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Written' f6r planners and practitioners in 

•employability skills development programs, this handbook provides . 
information and resources for 'designing aifd implementing 
competency-basea instruction. Chapter 1 provides an introduction to 

'basic terms and concepts that make up^a competency-based system* It 
describes four skills required for cne ta^be considered "ready to 
work" -or employable: pre-employment competencies, work maturity, 
educational skills, and occupational skills^ Chapter 2 presents a 
p-rccess and^feesources for planning a competency-based* eirployability 
skills development process^Techniq-ues are detailed fcr building a J 
vorkplan, gathering information to define competencies, laying - out 
the s6t of competencies and performance indicators,* and reporting the 

•process. Chapter 3 focuses on assessing competencies, lepics .covered 
include purposes for, assessment, methods, instrument development or 
selection, and organizing the assessment process. Chapters on 
program management considers developing management, goals^ Chapter 5 
offers a model for individualizing employability training. Ixampl^s 
of individual planning fqrms are provided. Chapter 6 lecka at 
're'eor dkee ping t and reporting, including audiences and*data sources. 
Selected forms are provided. Appendixes, amounting to approximately 

-one-third of the handbook, include a listing of resources, sample 
competency 'development plan, sample competency sets, and starter 
lis^s of published instruments. lYIB) 
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EXECUTIVE SUMMARY 



The most innovative programs that prepare people for work in the 1980s 
have used one or njore of the following, approaches: ft 

a ' v ~ * 

o Local emplbyers/ union representatives/ training fexperts and 
r other knowledgable persons have helped decide the essential N 
skills one needs to be hired and to succeed in a particular 
^ occupation. 

o * These minimum expectations are clearly stated for 'all to see in 
terms" of competency, levels a person should be able to 
demonstrate • 

o Learning activities* for the program are built after the desired 
outcomes are identified. 

o* Each person enrolled has a plan that is tailor-made to meet 
individual needs and aims. 

* 

o Time spent in various activities depends on how long it takes to 
acquire competence. 

i r V 

> ^ o Assessment techniques not only measure progress but continually 
emphasize application in real 14-fe situations. 



o Assessment techniques are varied depending on the type of skill 
involved. 



These jpgpgrams build new relationships and strengthen* ones that were not 
so visible before: 



The | youth or adult being served learns to become more 
self-reliant by helping to -plan each learning activity so it 
achieves desired ends* 

The youth or adult comes to view the community as a rich 
resource for learning both life- long and jobrrelatjed skills. 



o Relationships between theory and training are aade concrete 
through experience.- 4 

o Agencies become more closely linked together in a common cause 
to solve employment problems in the local area. 

This handbook was created through a joint effort between the U.S^ , 
Department of Labor, the National' Institute of Education, the Northwest 
Regional Education Laboratory/ and 14 local or state programs that 
initially developed and tested most of the ideas. 

Chapter One describes how being "ready to work" (or employable) .requires 
four kinds of skills: 

, in ' ' 



• 1* Pre-employment competencies include being aware of how the world 
of work operates, how' to narrow career choices, how to figure 
out occupational preferences and then how to get a job"* 

2. Work maturity involves 'demonstrating ability to apply skills in 
the workplace by meeting such basic employer expectations as 
dependability, having a positive attitude and working well with 
other people. 

* * *■ * 

3* tSincp most jobs require us to read, Vrite, speak, think, listen, 
or compute, there are numerous ways to apply the basic tools 
which .schools havejaeen dealing with for years : educational 
skills . Unfortunately, manw persons never see^ : the link between 
* • academic learning and how thpse fundamental skills 'are applied 
in everyday worklife. 

4. Knowing how\to perform the 4 te P~by~ ste P details -of a particular 
job is classified as occupational skills . ' While experts have 
toiled many years ,to describe the component tasks 6f a certain 
job, each local employer yfcypically decides what is acceptable 
for that particular situation, tfhat the , job-seeker needs to 
show is a solid grounding-in the essential basics -and an ability 
0 to transfer and adapt these skilly to an ever-changing work | 
environment. * " . 1 

A key term used frequently in 'this handbook is "benchmark." While a 
competency in one of the four areas above refers to a skill Or something 
a person does, a benchmark refers to the standard of performance at which 
a person is expected to demonstrate that competency* If an employer 
reads k portf olio^ fchat indicates Person A has demonstrated a particular 
competency^at Benchmark Level 9 out of a possibly 10, hiring might be a 
good bet. v * ; ; > 

v . . 

Benchmarks (standards of performance) become an essential planning tool, 
.then, in developing individual career training* plans and in telling 
employers what this applicant_can do. 

Chapter Two provides a scenario of a typical plannii|cj process you might 
expect to find in programs that have adopted competency-based training. 
Putting together a workplan is the first step^and this effort must^ 
involve a diverse, group of interested persons. Making sure competencies 
and benchmarks are locally relevant is key. While there are many lists 
of life* and career-related skillsuavailable, support for your program's 
own set will occur if you* build and ratii?y "locally-established 
competencies anc) standards. . * 

Pour information-gathering techniques ar^ described .that have worked in 
other communities to define competencies and set benchmarks: open forum, 
survey ,v DELPHI and Q-sort. 

A model is provided for laying out the set of competencies and 
performance indicators. Chapter Two also suggests Methods for reporting 
the process to th^ .ccqamunity-at-large. 
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Assessment techniques require careful consideration in competency-based 
programs. Chapter Three makes a strong plea for program staff to know 
their purposes before deciding on, any particular assessmfent strategy. 
There are essentially . three types of assessment activity: initial 
assessment at program entry; assessment used by instructors to determine 
an individual's progress/ and assessment -for external reporting. 

Depending on the competence being assessed, a variety of methods can be 
used: behavioral observation, product reviews < and oral or written 
questions. Each approach has its pros ahd cons for any particular 
purpose and Chapter Ttyree helps you sort out those variables for 
yourself. At one time or another you' 11 .be faced with preparing a 
measurement instrument yourself, so guidelines are suggested for 
designing -checklists, rating scales, open-ended questipns and 
, multiple-choice questions. Criteria are also provided for selecting ' 
existing instruments. 

The Appendix .provides a starter list of san^le instruments in** the four 
competency domains. Chapter Three discusses ways to organize the / * 
•assessment process, staff implications and the use of computerized 
testing are explained-. References $or additional information are 
presented. 

A competency-abased program needs to focus not only on how well 
individuals are reaching their goals, but whether the program itself is 
meeting, expectations. Chapter Four points out how management goals must 
be established by program staff ,' collabdrating agencies, advisor^ bodies'' 
and funding agents. Keeping ccmmunica^ion lines open is a constant 
challenge. Checking, reporting and .ad jilting program performance 
requires managers to seek out information" regularly and with clear intent. 

Chapter Five offers, an ideal model fpr individaalffcing employ abilfty 
training, while recognizing that to do it right requires intense 
interaction among people and trust* between agencies. We focus on the 
individual career plan and the recordkeeping system.* . Negotiation between 
the person being served and staff .typically follows a four-stage 
process: (1) initial assessment of needs, interests and 'abilities; 
(2) prescription of the appropriate learning experiences leading to 
competence; (3) ongoing assessment to' determine progress and to help 
.decide where more work is needed; and (4) eventual integration of theory, 
practice and insight so that the person "makes sense" out of \he -general 
mix of activities and can adjust the plan, if needed. Examples of 
individual planning forms are provided. 

Documentation and recordkeeping strategies can either be cumbersome or 
smooth, depending on your solutions to issues raised by Chapter Six. A 
first consideration is to determine audiences. Who needs what 
information to^answer what questions? Likely questions you 1 11 receive 
from internal and external sources are reviewed. Reducing staff workload 
must be a prime consideration. Selected forms from competency-based 
programs are jjrdyided. • < 



Ready to Work / presents some of the most promising developments of the 
last decade iji competency-based training. A number of exemplary 
Employment and Training projects have used these techniques with great 
success ip preparing an&v placing workers. Research has provpn the 
long-term merits of using performance as the measure of a person's 
readiness for ^jwork. > 

This handbook, then, is a resource for the future. It is not only a 
state-of-the-art look at career preparation~it is a working manual for 
planners as they shape programs in their communities. 




PREFACE 

Today, competency-based education is a true movement in both the 
secondary and postsecondary areas. It seeks to focus instruction on what 
the learner can do, and how that is best measured, instead of on how long 
the learner attends class, *■ < 

Compet6ncy-based techniques seem«to work particularly well in 
employabiiity programs because: 

o The skills are cleanly stated in'^berms of what the person' is 
able or expected to do. 

o There is an accented , measurable standard of achievement for 
each skill; this standard specifies in observable terms the 
' degree to which the skill is performed. 

There is a reasonably valid, reliable and useful method at hand 
for measuring attainment of the standard. 

o Instructional and training services exist which allow 

individuals to learn what is needed to perform at an acceptable 
standard. 

o The skills, at the specified standards, are accepted in the 
community as showing a satisfactory level of performance for 
entry-level employment. -„ \ 

It should come as no surprise that, these conditions are associated with 
successful learning: many practitioners in diverse educational fields 
'have found them to be fundamentally important. For instance, the 
technique of, task -analysis has clarified the learning of new jobs. 

~A recent study carried out for the U.S. Department of Labor (Michel et 
3l.# 1981) concluded that benchmarking , the method ijsed in the>CETA 
system to describe achievement of employment competencies, is the hext f 
logical step in improving employment programs. Those who have been r 

'involved in the use of benchmarking through the Consolidated Youth 
Employment Program (CYEP) , the Vocational, Education Demonstration Program 
(VEDP) and other DOL-funded demonstrations,* have become advocates for th^ 
use of- the concept. *• _ ' 

This handbook is written for planners^and practitioners in employabiiity 
skills development programs. * Its purpose is to provide information for - 
improving programs through the use of techniques of competency-based 
instruction, and. assessment . The handbook is arranged so that the user 
can focus either on topics of specific interest, . such as -involving tlje__ 
community or on the entire process of competency-based programming. 
■ 
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This handbook is .meant fQr use by people in roles such as: 

o Staff, counselors and program planners employment and training 
agencies serving youth and adults 

o Staff of agencies working to keep people out of the criminal 
justice system 

o Baployers wishing to take a more active role in workers* training 

o- Staff of vocational -technical institutes and skills centers 

o Career education specialists working with local schooJL districts 

o Counselors- and other instructional staff* in secondary and 
postsecondary institutions v , 

. o . Staff of adult basic education programs 

\ o Program staff serving handicapped learners and job seekers 

o, Vocational rehabilitation counselors' , * 

Itie handbook is designed to provide information and resources that will 
help you design and implement ""competency-based, instruction . It is 
intended to be flexible enough to meet the needs both of persons who need 
to implement parts of a system and' of persons who want to institute -a 
whole system * • • ^ *• * • • 

Some of the terminology in this handbook may be new, to. some readers; 
however, we have triea to offer enough illustrations so that the mearftngs* 
of terms, are clear. There" is nothing totally new in the concepts ^ 
presented here. * What is new is the way in which they are put together. 

The concept^ of competency-based instruction herald the next 'Step in 
training f oAemployafcility . By focusing an the employability needs of' 
tjie'indi vidua! participant, and by 'providing activities and assessment 
that can show) whether the needs have been met, these programs can 
consistently'prepajre people who are\ ready to^work. 

The authors of this handbook represent three distinct but closely related 
research and development thrusts at Northwest Regional Educational 
Laboratory (NWREL) Dr. Larry McC lure, and Dr. Greg Druian of NWREL* s 
Education and Work Program were responsible for overall preparation. Dr. 
Beverly Anderson of the Laboratory ' s Assessment and Measurement Program 
was responsible for the sections dealing with measurement and . . 
instrumentation issues . Dr. Robert Blunuof the Goal-Based Education 
Progriuh provided egi overall f ramework *f ar> v t;he design of. # • 
competency-focused system. ' jj t 
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x One:* WHAT IS A COMPETENCY-B^ED SYSTEM? 



I. 



OVERVIEW 9 r 



_K competency-based employment and graining system must rest on a solid 
foundatiop. Th,e one described in jthis handbook consists of four 
competency areas:' pre-employment ,| work matruity,^ educational skills and 
occupational' skills. Within each Wjrea, specific competencies' and 
standards are developed_which ref liect a local area or comnunity's 
consensus about skills which are needed *f or enqpldyability . This chapter 
provides an introduction to the basic terms and ccmcepts that make up a 
competency-iased system. 



II. FRAMfiWORK AND DEFINITIONS 



Before we look at specific elements of the system,, it will help to 
consider the framework underlying this gviide. Many frameworks are 
|>o3sible; the one* used here was developed by the U.S. Department of 
Labor. -It consists of four kinds of work-related skills which people 
*eed to have in* order to enter and succeed in the wtyk £orce. Thes§ four 
kinds of skills are Referred to as COMPETENCY^ AREAS . Within each* broad 
competency area, local employers select specific competences , indicators 
and apprqpriate standards of performance (benchmarks). >\. 

The four competency areas are: ^ * ^ 

Pre-employment . This 'cop^etency area covers a basic' awareness 

of the world of work, the range of career 
f alternatives available and methods fot 

selecting jamong the alternatives. ^ ' 

Pre-employment skills include knowledge of 
one's own career interests and^ of ^one 1 s 
strengths and. limitations ^as they* relate to ^ 
vaYious career choices ^ ^ 

Work Maturity . c A This competency area involves a demonstrated 

ability to meet the employer's expect^tldns 
< of basic employee responsibilities, e.g., 
attendance, ability to carry out instruction, 
, ' etc. ' , 

Educational Skills . 1 This competency area deals with mathematiSK 

y 1 *»d language skills appropriate to career^ 

/ [J- * interest*, and the application of these 

skills to real work 'situations. * 



/ Qcc national Skills . This cbnqaetency area includes mastery of 

basic and advanced* skills required for a 
specific occupation or cluster of occupations 

Within each of these competency- areas, a number of COMPETENCIES should be 
identified. Competencies are defined as minimum skills necessary for 
en$>lo£ment, and they differ from community to community. "'They max'be . 
Required, in a< variety of formal and informal ways, including schooling, 
pn-theYjob training and Life experience. In the context of enployfcbility + 
skills development, competencies refer to more than the occupational 
qualifications to obtain a specific job — they also reffer to the cluster 
of related sk'ills that enable a person to continue to compete in the * 
labor market. \ ~ 

INDICATORS are behaviors thafshow whether a particular competency has 
r been mastered. For example, demonstrating career awareness might be one 
of the pre-employment* competencies. To indicate mastery of this 
competency, a person might name careers available in the community. ' 
Sometimes a conpetency has many indicators associated with it and 
sometimes ipnly one or two. Also, it, may not always be necessary to 
accomplish every indicator in order to show mastery •of a competency. For 
inStance> the competency "having a positive work attitude" may -be 
demonstrated by indicators such as "seeks ways to inprove performance" or 
"takes initiative in finding* tasks." Performance on one of those 
indicators may be sufficient to indicate mastery of the competency. 

The important poifct to remember is that a competency describes desired 
skills, knowledge or attitude^; indicators describe the observable', 
measurable behaviors that your program will accept as evidence that the 
competency has b£en attained. # * 

* STANDARDS OR BENCHMARKS are closely related tt> indicators. While an 
indicator refers to a behavior, or something measurable a person does, a 
standard states the textent to which a person is* expected to demonstrate 
the indicator. s : p 

Indicators and standards are usually stated together. For instance, 
under the competency of ^demonstrating regular 'attendance, " the indicator 
"show up on time in a* work experience placement" is joined with t-he v ^ 
standard "four out of five times." 

-There, is another reason starfdards are a useful^ device for measuring 
employment-related skills: If set locally, they a^re accepted by 
employers in the community as minimal acceptable levels of performance 
for entry-level jobs. This means tha£ a person who has achieved stated 
standards in all four competency areas is likely to be successful in an 
entry-level (or better) position. 
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R6bert, who lives in a large city, types 34 words per minute • and he 
aspires to be a word processing specialist* In the Occupational Skills 
Competency Area, however, the standard for typing is set at^65 wpm for a 
person wanting to-Tnter the word processing area. Robert knows that he 
will have to nearly double his speed in, order to get a job. But Carla, 
who lives in a rapidly growing suburban area, could qualify by typing 
only 55 wpm because the employers in her area, faced with more jobs than 
people to fill them, set the standard lowe* to make the job easier to 
fill. ' * 



Competencies, indicators and standards (or benchmarks)* are three of the 
four major .concepts in this system. The fourth is ASSESSMENT . In the 
pasti assessment (testing) had little to do with what a person actually 
did on the job. As will be explained in Chapter Three,. the function of 
assessment becomes extremely important in competency-based systems. 



Orlando enrolls in a computer tdass. At the outset of the class, the • 
instructor sets forth the competencies each student will have to master 
in order to complete the class requirements. 1 At least one instructional 
unit is devoted to each identified competency. Attainment of each 
competency is .measured by using an assessment strategy designed .for the 
purpose. 



In a competency-based system, assessment becomes the way to document and 
verify whether a person has attained a given standards Or, if 'the person 
has not attained the standard, the assessment process khows what remains 
to be dpne to attain it. 



Catherine is a photographer. In order for heaf to advance in her 
profession, she has to assemble a portfolio of her best photographs, 
which she then takes to prospective employers. The employers assess her 
work in this way. (She does not have to take classes or pass a licensing 
test ,ih Order to grow in fi 'her field.) ** 



♦"Standard" and "Benchmark" are used interchangeably in this handbook. 
Both terms refer tcft how well a^person performs the competency. * 
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Con^etencies, indicators/ standards and assessment are dependent on 
one another in a smoothly functioning system. Socn'e exqrpfiles m&y 
illustrate different ways the relationship occurs: 

Indicator and Assessment 
Competency Area (Minimal Skill) Standard . Method 



Work Maturity 



The participant will 
demonstrate regular 
attendance • • • 



by showing up 
on time four 
days out of 
five in a work- 
experience 
placement • • • 



as measured by 
an employer 
checklist that 
includes an 
item on punc- 
tuality • 



Pre-employment 



The participant wi^ 
demonstrate career 
awareness • • • 



by naming five 
careers avail- 
able in the 
community • • • 



in a counsel- 
ing interview. 



Educational 
Skills 



The participant will 
communicate through ■ 



waiting 



t 



by preparing a' ' 
brief business 
letter that is 
neat, and con- ' ' . ' 

.tains no more 4 
than one gram- 
matical or spell- 
ing error • • • in a classroom 
exercise 
simulating a 
business 
environment • 



Occupational 
Skills^ 



T£e participant will 
perform filing tasfcs 
by • • • 



filing materials 
alphabetically 
with no % 
mistakes . . . 



given a. paper, 
and pencil * 
test" with sev- 
eral items to 
alphabetize* 



» . 
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Within each competency area, a number of competencies will usually be 
identified, along with indicators/ standards (benchmarks) and a method' of 
assessing whether the standard has, been' m^t. 



III. HOW COMPETENCY-BASED SYSTEMS t 

CAN IMPROVE EMPLOYABILITY PROGRAMS 



Competency-based systems offer a way to (1) closely tie "the program to 
community needs and tequirementa^-vt31 provide fof~~ program and individual 
accountability and (3) ; coordinate all program activities, from planning 
and assessment through training, counseling and placement. 



: — : ! * ? 

In one program for yoiing people operated by a CBTA prime sppnsor, Jean 
was given tests upon entering the program and the results' were used to 
select a work-experience placement for her after school. In the 
meantime, her school counselor had given her >a* career interest 
inventory. But even though the »CETA and school counselors at times 
worked together, they did not share the results of their various' 
assessment activities. > Furthermore, Jean's work at school did not seem 
to contribute, to her, performing effectively in the Job— and she did not 
see how her job tied in to school work. .When she graduated from high 
school and had^job interviews, Jean elt she didn't have a Way of showing 
wfiat she could really do. 



Because a competency-based system is tailored to the needs of the 
community, and because it reflects local standards, participants are more 
likely to be acceptable to employers. In one area for instance, the ■ 
appliance store owners majj be willing to employ persons certified by a - 
community college as having mastered competencies in the area of 
television repair. They may be less willing to hire a person from 
outside the. area because they are unsure of the standards to which the 
person was trained. -In the same way, competency-based systems can report 
not only* competencies in occupational areas, but ^Lso in areas of work 
maturity, pre-employment and education. Just as a person is?able to 
approach an N employer with evidence of ability to fix ^T.V. set, a person 
should also be able to provide evidence of reliability, of basic 
educational^attainment and other attributes the employer may be looking 
for. The strengths of the system are that: 

o aaplqyers have participated in the setting of- standards^- 

p Attainment of standards is demonstrated in concrete and. measurable 
terms. * 

o Skill attainment ±9 demonstrated in all areas related to work 
success— not Only occupational skills. 
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Competency-based systems also, foster accountability by providinc 
, individualized instruction based on mutually understood goal^T In an 
example used earlier, the person who coul£ onlyj type 34 words per minute 
knew that he would have to type approximately 30 Wpm faster in order to 
get an entry-level job. 

'In a compet€ncy-based system, it is important whether you can demonstrate 
a -skill — not how you learned it. Indeed, different people learn in very 
different .ways. Moreover, the kipds of skills needed for work place 

.success are .gxt^emely diverse. By focusing on standards, individuals are 
encouraged to develop skills in the, way that is most appropriate for 
them. As a consequence, 'participants.dn such systems tend to be: 

* • 

o Working toward clearly stated goals 

o Clear about how what they are £oing relates to achieving the gpa! # 
b Working towards goals that are meaningful and important to them. 

.A ^competency-based system can also strengthen programs because it creates 
linkages among agencies and institutions which offer related services. 



A youth services agency established a "benchmarking" system for its high 
school age clients. Not havirfg the staff expertise "*to offer instruction 
in basic skills? this agency accepted certain school, credits as evidence 
of attainment of benchmarks in th£ Educational Skills^rea. Furthermore, 
because the v hj^h school in the arjea offered a career education course 
that covereaTOmy of the -benchmarks associated with the pre-employment 
area, the agency also accepted 5, credit from that course for appropriate 
benchmarks. The same agency also learned that a juvenile justice program 
serving certain >high-rfsk youth al^o focused on pre-employment skills. 
Youth % who-participa ted in that program were able to gain certification of 
appropriate benchmark attainment. 



: : / ; 

Of course, acceptance of another agency's program doesn't necessarily 
guarantee that benchmark performance n*ll be attained. *But by finding 
out what other agencies offer, and by entering into dialogues with staff 
of other agencies, it may be possible to dramatically increase both the 
coordination of services and the ef?icien'cy with which the services are 
delivered. * 

* . " a 

Putting It Into Practice ^ 

There are varying degrees to which one can develop the concepts 
underlying competency-based eraployability programming.* The ten points 
listed below are intended to stimulate the reader into thinking of 
alternatives which could be used in program planning: 
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All Competencies Can Be Worked On Simultaneously 

The four competency areas are not necessarily sequential although 
services could be. A participant can and should Mke progress^in 
each competency area simultaneously without meeting prerequisites. 
It any competency target is first, it would come under 
Pre-Employment . 

Interrelationship Among Competencies 

Some competencies can be acquired through working on other ^ 
competencies. For example, ^ participant uSes critical thinking and 
problem-solving skills v in miking occupational selection or mapping a 
strategy for job search (in effect, getting "credit" fdr ^ach). 

Individualization Requires Negotiation 

The individualization process which is the cornerstone of a 
con^>etency-based system requires a willingness to "negotiate" with 
the participant^ and consider alternative paths to completion. 
t Participants develop botK^n ownership and a better understanding of 
prescribed competencies through this process. 

Specific Career Focus of Each Competency 

Career interest and active worksite involvement must "drive" 
acquisition of competence in all four areas. Preparation for 1 
competencies must not be an end in itself or conducted in 
isolation. Ir\ other words, a specific participant-identified career, 
or^ob context should be the organizer for each competency-based 
learning activity. This wbuld mean that Education Skills would be 
f Acquired in tandem with other program activities, particularly 
oriHsite wqrk ^cperience. * 

.Community Identification of Competencies/Benchmarks 

The local community must hel£ identify or ratify the competencies in 
each- arw<and perhaps suggest additional ones. However, arrivin£ at 
'a^ingle acceptable standard or minimum competency level seems 
difficult. Each employer should be ablfe to make the judgement if 
the participant's ^>rc»ress along some kind of scale is' acceptable. 
For example, to say tRat all program completers who show typing 
skills on their records will have 50 wpm may not be necessary for an 
employer who will accept 45 wpm with higher accuracy'. 

i 

Challenging Competencies 

If a participant can demonstrate a competency without an 
intervention, that should be encouraged. Persons witfy previous 
skills and experiences can receive an inmediate "rewaird" without 
going through unnecessary hoops. 

* • 111 
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Coag>etencies Are Not Time-Based 



Each participant is likely to take varying amounts of time to 
prepare for, practice/ and demonstrate a competency. 

i 

8* Assessment Approaches 

The assessment approaches should be selected ii^order to measure 
identified conqpetencies . Assessment needs to meet the purposes for 
which it is designed. No one assessment approach is good or bad in 
and of itself. The purposes always need to be clear. 

9« Participant Self-Assessment 

Assessment should always include the participant '3; self-assesgrfen* 

10 • Community. Certifiers 




It is possible and often desirable to use employers in the ^fepwrnanity 
as certifiers of con^etencies that participants' acquire o»*€he job. 
Jf this technique is used, employers should receive orientation and 
training. 



IV. SUMMARY 



Competency-baped systems ,are coordinated, comprehensive "programs for 
defining, assessing, attaining and reporting skills needed" for success in 
the workplace. Many agencies and institutiohs have employability 
programs which suffer from a lack of interrelatedness iamong the various 
activities. % Consequently, employers may -receive an inaccurate idea of 
what the participant can do; participants may not be able to do^what 
enqplqyers need or exf>ect;* and, while in the program, participants may be 
receiving inappropriate services. 0 
*• 

Competency-based systems avoid such problems by focusing qn the full 
spectrum of needed skills^ by defining those skills in terms which are 
accepted by enployers,* and jby^providing individualized assistance in 
achieving specified performance levels. ■ Competency-bAsted systems also 
lend themselves extremely weir|to. coordinated services with other m 
agencies. 9 

Appendix A of this guide cpntains a resource list of ^persons, programs 
and agencies which might be contacted for further information on 
competency-based systems. \ - 



Chapter Two: SETTING AND COMMUNICATING COMPETENCIES 



OVERVIEW 



The purpose of this chapter is to present a process and resources ^or l 
planning a competency-based employability skills development program. s 
The chapter begins with a suggested- scenario of the planning process. 
Next appears a snption whicb details techniques that can be vised in / 
carrying out each step of the planning process. Appendices contain a 7 
, "starter lisV* of competencies in four areas (pre-employment skills, work 
maturity skiir^, educational and occupational skills) and a san$>le plan 
as it might have\been developed in a particular area. 



II. A\SCENARIO OF THE PLANNING PROCESS 



This planning scenario takes place in a rural setting where the balance 
o^ state prime sponsor for CETA programs serves a very large geographic 
area. The area includes several small towns and the main economic 
activities are farming, ranching and mining. While the\CETA program 
serves both youth and adults, .there has been a priority on employability 
development for youth betweeA 14 and '18 years of age. In addition to the 
emphasis that* CETA places on youth programs, schools in some of the small 
towns have begun to include career education in their curricula and most 

of the high schools offer vocational education to their students* 

* > 

A 'competency development effort, was initiated by the youth coordinator . 
for the prime sponsor,' with support of the Youth Council. The first step 
was to identify all agencies that provide employ ability development 
services or programs to youth, or that 'have an interest in doing so. 
Telephone qalls to schools revealed an interest by counselors and, 
vocational educati6n teachers. . Calls to regional employment service 
centers\resulted in interest by staff in some of the centers and 
referrals to other agencies such as Chambers of Commerce, Kiwanis Clubs 
and the regional econpmic development council. Other contacts weye made 
and an initial list of interested individual was compiled. 

The liijit included individuals representing the following groups: 

o High school' ^nd junior high school counselors \ 

tt o High school and junior high school business, economics and s 
industrial arts teachers . \^ 

o High school and junior high school administrators including a few 
district level administrators 

o Chamber of Commerce 
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o Service clubs such as Rotary and .Kiwanis 

o * Employment service t 0 t 

o -Regional (within state) economic development council 
o Employers who hire youth • 

o Corrections staff ' ' , 

o Organized labor , 
o Migrant representatives 
o Other minority group representatives (e.g., Native Americans) 

* 4 To ensure balanced representation of various groups interested in -the 

project/ the youth coprdinator added two people to -the youth advisory 
^6o&l}Cil and asked the group to*serve as the planning team for competency 
development • The two people added were a person from a Chamber of* 
Commerce and a high school teacher. Other groups were already 
represented on the youth advisory council. ^ 

After discussions yith the youth advisory council chairperson, the 

# competency development project was put on the agenda for the next 
meeting. The youth coordinator agreed to find several samples of 
competency develbpment efforts to review f or fche cotacil as a starting 
point. 

/ 

The coordinator's search for other competency development efforts * 
included contacts with the state employment service, several other CETA 
. pxrime* sponsors, the Office of Youth Program^, Department' of Labor, the 
state career education and counseling coordinator, a regional educational 
laboratory, the Regional Office of the Department of Labor, and 
administrators and counselors fromwa few school districts. Individuals 
contacted were^Sjjced the fdllowing questions:. 

o H^ve y<>Ut developed competencies and/or ^oaJLs and objectives 

related to empfoyabi lity development? If so, how did you go about 

itr ; c * - ' ~ - J 

o Do you have .written -material describing how you developed* the 

competencies and/pr the competencies themselves? If so, will' you , 
sfcnd a copy? „ * 4 • 

> % o Do you "know of any competency development efforts that are * 

exemplary and that we should' review? If so^ who is the person to 
contact? * 1 

The coordinator ^earned a goodt deal about processes during the 
conversations, and received several documents listing and describing 
emplbyability-related competencies and/or gpals and objectives. 

The major* objective. of *the advisory council meeting was to introduce, the 
competency development pr ojec t and form a. draft plan for the effort.. The 
youth coordinator -and the advisory council chairperson decided that the 
plan shoiilJfr^nclude the fqllowing|sections: 

—J 

Jit* 

• U.: • 
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o Objectives, .for the effort * * 

o Strategy for accon?>l±shing th<^work, including the information 
to be^ collected, techniques for collecting information and 
procedures for analyzing and summarizing the information 

Q \ A list of individuals and groups from whota information would be 
^ collected, and techniques for "stapling 

o- A* description of the anticipated end product 

A description of how the competencies would be communicated to all 
interested parties and used by the- CETA balance of state prime 
sponsor ^ * 

o A listing of resources fo^the effort 
o A schedule for project «ctivit*s 



o 



The yoi^th coordinator was asked by the council chairpersoa to prepare an 
introduction for the council that included a review of information 
collected from other con^etency development efforts, a description of 
iliformatioiv collection technique^, a description of several possible 
formats for the end product and an initial statement about resources for 
the effort'. ' 



> 



The youth advisory council meetings were four hours iofcg. This allowed^ 
council members to drive in, meet and drive bacJcin a single day. The 
^outh coordinator and the advisory committee chairperson' decided that two 
and one-half hours would be t devoted to planning the competency 
development effort and that it would be the top priority item for* the 
meeting. r - - " _ > 

* « 
The youth coordinator introduced the competency 'development effort. She 
9 ^stated that there is a need to. firm up the purpose of youth em£loyability 
tarograms: ."Respwces are becoming scarce and we mustjbe sure that youth. 
Wre gaining skills, knowledge and attitudes needed to 'become valued 
employees, there *is a need to be clear about what youth programs will 4o 
and to assess- the extent to which important cbmpetefccies are achieved by ! 
, £hose who participate in the program* 'it fs very ifoortant to be able to 
: * explain to youth/ employers and othfers what the protfrajr does so that 

there is no confusion or unrealisitic -expectations. It is al^o $m>ortant 
•to have a good, clear set of competencies so that various agenciefc * % * 
serving the same youth can cooperate in providing a^ full eimi^ability 
development program rather than overlapping and duplicating services. 
Developing a clear, compl'ete. set of -competencies will improve 
communication among agencies and individuals, "reduce confusion about 
youth employability services/ provide the basis for wise use of scarce 
resources and bring better accountability to the programs. w 

>. 

The youth coordinator went on to describe^ the various agencies and 
V * s organizations in the area which either provided emJ>loyability services or 

were interested in doing so. She also gave a' brief review of competency 
development efforts from other areas and described 4 several possible data 
collection techniques. She showed examples of how the final product, a 
« list of conqpetencies and standards', might 16ok. 



The chairperson then opened the meeting to general discussion about the 
project. Aftfr a few minutes of discussion the grox^p reached consensus 
that the project would be valuable and that they should proceed with the- 
planning. . 

/ 

The chairperson described wjiat the plan should' include and opened a 
brainstorming session on specific objectives for the study. The groufi 
proceeded through each section of the plan, using a process of 
brainstorming, clarifying, evaluating, and prioritizing i^eas. The group 
ai^t time limifcs x so that all .sections of the plan could be discussed * 
within the agreed two and one-half hours. 

By the end of the session, the 'youth coordinator had enough information 
to prepare a draft plan. t She: agreed to draft the plan within the next 
two weeks and to send a copy,*to each advisory council member for review 
and comment. She also agreed to compile the results of the reviews and 
to prepare the final plan. (Appendix B contains the plan that resul-fee^-^ 
from this effort.) s 



III. .BUILDING A PLAN \ 

"» < 

Key Points 

A competency-based employ ability development system must start with broad 
consensus on what it tAkes to get and hold a job. Establishing the * 
competencies to be" learned through program activities is basic to the 
ultimate success of the effort* Haying simple, clearly stated 
cong)etencies lets those receiving service, those providing service, 
emplo^rs and. the community at, large know exactly what is being delivered 
through the program. A broadly accepted set of cjgnpep encies helps 
several agencies share responsibility for employability development of 
individuals.. Each agency may concentrate on a portion of the 
can£>etencies with full employability development resulting from the 
combined efforts . This is efficient, it -improves cooperation and it 
reduces overlap. And a good'set of competencies allows programs to set 
and assess performance go^ls and to communicate the results back to 
interested individuals and groups. Accountability is achieved in this 
way. , 

t 

Planning 

Before setting out ,to establish competencies, a plan should be developed 
by agency staff. ,Staff should seek input from interested groups, s 
including -those proyiding service, those receiving service, employer^, 
affrd the community at large. Existing advisory committees might also J 
provide needed input. 



J 



A useful plan usually includes categories like the following: 
o Objectives of t"he effort a^d expected products 
Procedures to be used in completing the work 

Re^>onsibilities and authority of those who arerto participate 
Resources required to complete the effort 
Schedule of activities including completion dates 
A description' of how the results will be used 



Details of the plan should be chosen to. fit local condition^. The plan 

fhould outline procedures that wj.ll meet local objectives, within the 
imits of available 'resources. / ^ 

In preparing a plan, consideration should be given to the following 
general step's : , 



COLLECT 
INFORM INFOR- 
♦PEOPLE RATION 



DRAFT 
INITIAL /' 
COMPETENCY 
SET 



DRAFT COMMUNICATE REVISE 
VALIDATE FINAL THE AND 

THE SET SET RESULTS UPDATE 



Informing people about 
the effort. 



Collecting information; 



Drafting the initial 
set of coiftpetencies . 



This is a crucial step — people Who know 
about 'the effort are /likely to, participate 
in it. The step iftcfludes preparing general 
information about the effort and getting 
'the information to those people you want 
. involved in the work. It may involve news 
releases, radio and/or TV announcements, 
targeted mailings,* presentations and other 
techniques. 

, This step includes organizing the actual 
wotk group preparing descriptive 
information and instruments, carrying o\it 
the planned data collection activities, and 
organizing and analyzing the data collected. 

In this step th6 data collected are 
transformed into« a^simple, easy to read set 
of competencies. It involves choosing 
criteria for including specific 
competencies in the set, designing. the 
format for the competency set, and writing 
and rewriting statements until they are ■ 
easy to understand. 
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Validating the competencies * 
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Preparing the final set . 



This step involves having constituency 
groups review the initial s€t of 
competencies to make sure that the 
competencies included are the important 
ones and that the statements are clear. It 
is also possible to prioritize competencies 
during rhe validation process. 

With information from the validation 
process as the basis, the final, set of 
competencies' is written, tfhis involves 
deleting;' rewriting and reordering the 
competencies. Final editing and formatting 
is done and the competency set is prepared / 
for production. * 



Communicating the • 
competencies widely * 



Update periodically . 

r 



Perhaps the most important step in the 
effQrt is making sure interested 
individuals and groi^ps learn' abou£ the 
results of the effort. A campaign to 
" communicate should be conducted through 
news releases, radio and TV announcements, 
presentations^, targeted mailings* and -other 
techniques. People should be^ told about 
the processes used to establish the } 
competencies as well as the content of' the 
competency set* 5 

Competency sets should be reviewed and 
updated at least every three years. 
Results of -the use of competencies should 
be communicated to those who helped develop 
them. Conditions and needs change and 
these changes should be reflected ii* the 
competency sets. There 'may be additions, 
deletions and revisions- New priorities 
;may appear also. The updating process will 
probably not involve all of the same 
activities that resulted in the original 
set, but enough should be done to assure 
good input from key groups. 



1 



The processes described above are those that a local community should 
follow to establish a locally* relevant set of employability 
competencies* A good preliminary step is to identify and study 
competency sets developed by others. These sets of competencies provide 
a starting point* 'They introduce ideas, and help the thinking of planning 
'groups. Finding and using the work of others is efficient and Will 
probably improve the end result of local efforts* Resources listed dn 
the Appendix can help you locate other competency sets* 
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Involve tflfie— ftfght People 

An hypothesis about setting competencies might be the" following:; The. 
greater the involvement in the development process, the greener the 
acceptance of the end product. R * « ■ 

Having conipetedbies written by a single person or even a' small group is 
fast and easy, but widespread acceptance is unlikely. On the other )ia nd * 
involvement of * large numbers of people from diverse - groups is cumbersome, 
and makes consensus difficult; yet reaching consensus as competencies are 
developed will virtually assure acceptance of the final product. Local 
conditions and resources' Vill determine who should establish competencies 
and how they ^hojild be involved. * ' » N 

As a plan is' developed and carried out, consider the following groups as 
potential sources of input: 

o Agency staff * s 

o -^Staff from other agencies that provide employability services N 
♦ o anployers— tho§e)(people who wijLT?hjre products "of the prografii 



o Representatives from various corhnrcmity organizations that have an 

interest in employability development, such as the Chamber of 

Commerce, I Kiwanis, Church related, organizations, YMCA, YWCA, 

Scouts, Urban League', etc., * ^ *. * J ^ 

* * * 

o » Individuals who receive services 

o* Councils and advisory committees — Private lndustJ^Co(fncils, * 
caift£r and vocational education advisory committee^, youths-/ 
councils, advisory committees and §tate and local employment 
service agencies 

It is .not only inqportant to involve the rigA peoples-it is important to 
decide on specific ways in which various individuals knd groups witljbe 
involved. Will "they , brainstorm competencies? Will they review^/md 
validate? Will you use a few representatives or the whole grpujfr Will 
they make decisions or give advice? Will 'they have responsibilities^ f or 
data collection? Will they have authority? , v * 

The roles, responsibilities and* authority, of each individual and group ^of 4 
individuals must be decided and communicated. Expectations must be cleax 
and acceptable to those who are, involved. 

\ r . ' * 
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IV. COLLECTING THE DATA 



Gathering information — Four Important Techniques s 

While it is possible for one person to Research the literature and 
develop a good set of employability competencies, the result of such an 
effort may not be accepted by key individuals and groups because they 
were not, involved in <the process. This section will describe four, main 
methods f 4 or involving people through data collection. 



Choosing the Right Technique 

There is no single best approach to data collection; the best method bfc 
combination of methods is the one that suits the local situation. 
Choosing the right approach should be based on consideration of the 
following factors: p <* 



Purpose of the information 
gathering effort .' 



Staff 'capabilities and time . 



People to ^provide the data . 



t The main purpose is to get the 'information 

needed to prepare a good_se£„ of 

competencies. Other purposes may include 
developing awareness of program efforts,* 
building support for the program and 
establishing dwnership for the competencies 
developed, if information is the only 
purpose, %he techniques .used will probably 

• be different than if information and 
support are key purposes. 

It lis necessary to determine whether staff 
members have the background and skill to 
plan an£ conduct dat^ collection efforts. 
Can staf^p members develop a questionnaire? 
Can they facilitate brainstorming 
sessions? Can they analyze an& summarize 
data collected in a survey? * Another 
important consideration is whether* staff « 
m&abers have time to devote to data 
collection efforts. How will w&rk on data 
collection 4af feet other important task^? 
^Jan^ing an elaborate data collection 
process when s^aff has little or no time.to^ 
work on the project is unwise. 

Consideration "should be given *$:o the number, 
of different groups from which data will be 
collected and to the number of individuals - 
within each group to be queried. ' Few 
groups and small numbers should be handled 
differently than many groups and larger 
numbers. * " * 
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Time schedule for 
development • 



If the competencies. are needed immediately 
the techniques will be single- and 
straightforward. If there is'mbre time the 
procedure can be more comprehensive. Think 
about how lo^g things take. Allow a 
generous amounfTof time as work usually 
takes a little longer than estimated. 

Is there an existing se^of conqpetenciels < 
that can be validated or is this a new 
effort? Validation can be done one way 
while original development will take 
different techniques. 

What funds / staff time, volunteer services 
and time and other resources are available 
to the project? Can consultants be hired? 
Is there money for printing? Can a trained 
facilitator be found who will volunteer 
services? Be sure to know all resources 
available as they will determine techniques 
to use. 
>\ # 
The local competency development planning' team should consider each 
criterion ,and choose data collection methods that are appropriate for the 
situation. * * 



Current status of 
development. 



Resources available. 



Four Methods * ' 

Choosing the data collection techniques best suited for local conditions 
is not always easy. There are. many methods that can be used/ but four 
.will be discussed in this section. Adaptations and combinations of the 
four methods give a wide range of possibilities. The four basic methods 
follow: 



Open Forum . 



Survey. 



DELPHI. 



This technique involves face-to-face 
interaction of people in groups. Group 
sessions are usually conducted by an 
impartial facilitator and the intent is to 
gain consensus among group members. 

Surveys are usually paper and pencil 
instruments 'on which individuals express 
their opinions about a number of statements. 

This is a special type of paper and pencil . 
survey that results in group consensus on 
various statements after three rounds of * 
responses. It is a technique through which 
consensus is built without face-to-face 
interaction. 



r 
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%~ SoTtm Shis technique involves rating and/or rank 

ordering a group of statements according to 
- importance. This is accomplished Ijy 

sorting a deck of cards in which each card 
contains one statement. Once the sorting 
y is complete, the participant recordfc^the 
relative iaqportance or rank on a response 
sheet • 



OPEN FORUM 

The open forum approach to competency development involves one or more 
groups of people defining a cceqpetency area, competencies, performance 
indicators and standards through open discussion. The group discussion 
is usually conducted by a person trained in, group facilitation, in this 
approach, the definition comes» primarily from the background and biases 
of the people involve^InEhe discussion. The group usually starts with 
a blank page and woi^s thr6ugh brainstorming, clarification, 
prioritization and standard-setting steps. The result is consensus on a 
set of competencies. A major benefit is ownership of the set of 
constancies developed out of intense involvement in the development 
process. The major disadvantage is that the. competency area ma/ not' be 
designed comprehensively because of limited background and narrow biases 
on the part of the participants. 



Brainstorming . 

1. u Introduce the topic with a brief description of the four 
* competency areas and give .the rules for brainstorming. 

2. Set the expectation. that the gzGup \*ill be working within one (or 
more) competency areas. 

* * 

3. Ask people to give their' impressions about the needs (or — 
failures) of potential clients in the competency area under 
consideration. Clients may be youth or adults and may be 
handicapped, i^lnorities, etc. 

4. - As individuals talk, write key ideas on a large sheet of paper~ 

and paraphrase key points after each individual talks. 

5. After each individual has made a general statement, " ask the group 
to describe some skills, knowledges and attitudes the clients 
should demonstrate in the area. 

6. List all ideas without evaluating them. Be accepting of input 
' from everyone. Sjaek input from" those who seem hesitant. 
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Clarifying * 

7. When the brainstorming has slowed, go back through the entries 
and get additional explanation and description as needed so that 
everyone has a common understanding of each entry. 

8. Attempt to condense, simplify and combine without losing main 
ideas (important competencies) • 

9. Rewrite competency statements as necessary to bring them into 
good understandable form. 

10. Continue to work at this process until concensus is reached. 



Prioritizing 

11. Ask individuals to'decide which of the competencies are the most 
important and to present their choices and rationale. 

12. Ge t opinion s from e veryone and tally^ opinions _pn Jthe chart* 

o 

13. Continue discussion until consensus is- reached on the top seven 
to ten competencies. 

/ 

4 t 

Setting Standards 

14. Taking the competencies one at a time, ask the group to list the 
behavior clients should exhibit if they are to be considered 

competent. This will result in performance indicators. 

~ * <$> 

15. Repeat brainstorming, clarification and prioritizing steps for 
each competency unt^Ll consensus is reached on. performance- 
indicators. * 

16. When performance indicators have been agreed upon, ask the group 
1 to list standards (benchmarks) for each indicator. > * 

17. Repeat the processes until consensus is reached, on standards. 

The open fpinim approach to setting competencies may be time consuming. 
It may take' several meetings to refine areas, list competencies and . 
performance indicators and set standards. The time in between meetings', 
however, is useful in clarifying ideas and issues. It may also be a- good 
idea to work with several smaller groups and synthesize the results into 
one set of competencies. 

In considering the use of an open forum for data collection, several 
factors should be kfcpt in mind: ^ ^ 
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Participants must be 
prepared . 



People who coma together for the fi%st ^ 
time mu*t go through some group-building\ 
exercises before they will work well 
together. They must get to know each 
other, learn the perspectives and biases 
of ^various people and begin to feel 
comfortable with others in the group. The 
group must also be introduced to the 
purpose of the session and the expected 
outcomes. They must have a chance to 
question the effort, get answers and feel 
that their contributions will mate* a 
difference. - Preparation is very„ important 
ancjr^ good group facilitator knows how to 
get people going. 



Discussions can get .bogged 
down in definitions. 



In planning for an open forum, key terms 
should be defined and definitions shared 
with the group. It is ver^important that 
the group not stall because people * ^ 

disagree on ^defcijntiJ^i^ 

moving. 



Constraints on participation . 



Select people carefully . 



There may be some major ^constraints on 
participation by key people—employers, 
for example. People may not be able to 
'participate because of distance. Others 
maV^be able to spend only one or two hours 
inla' meeting when more time is needed. 
Still others may be able to attend one 
me^ting^ but "not a series of meetings. 

Those chosen to participate should agree 
to/ full involvement. This*" means that they 
will be ible to ^attend all meetings for 
•J the full time and that each will attend 
Jf personally rather th^p .sending a * 
/representative. It is also important to * 
| select- people who are open and have no 
major "axes to grind." Each person will 
have an individual perspective and biases, 
but must be willing to disten to others * 
and accept their ideas. 



Approach people carefully . 



When people are asked to volunteer their 
time to an effort such as competency 
development, they must be asked .in a way , 
that will make them respond positively. 
They must be given enough information to 
know what the project is about and have a 
clear understanding of what their* role 
will be. Also, and perhaps most 
impoS^mt,, they must be made to feel that 
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their involvement is important and that vft 
their contributions will make a \\ 
difference. 



The open forum, if done well, can re&ult in excellent .sets of 
cc«5>etencies and strong commitment to the competencies by those involved 
in the process. The people involved may form an advocacy group for the 
effort. \They can spread the- word back to others in the group they 
represent! • 

On the <5ther hand, .it is difficult to reach consensus when a group is 
composed of people from diverse groups. The biases may be too strong to 
overcome. It takes a skillful facilitator to keep a group moving toward 
common ei>d$. 

- . ./ 

SURVEY 

• * 

A survey is most often a paper and pencil technique for collecting data. 
Telephone surveys and face-to-face or grou$ interviews are alternatives 
to paper and pencil methods. , A mailed survey instrument consists of 

'directions and a number of specific statements to which people respond. 

Jfhe response is made by checking or entering a number* in space provided. 
The respondent shows agreement or disagreement or indicates relative 
importance or value of the statements. In seme cases, statements are 
rank ordered in a survey. The survey instrument is usually sent to many 
people in the mail; they are asked -to respond an<J Return the instrument 
via mail. ' 

Conducting, a paper .and pencil survey involves the following steps: 

Develop the survey instrument-^a questionnaire or opinionnaire. 

Pilot test the instrument to make sure it elicits information as 
expected. » 

Duplicate the final instrument. 

Identify the audience for the survey and get addresses. 

Send survey instruments. # 

Send followup notices to those who do not respond. 

Organize survey responses, compile data, analyze and summarize 
information. 

Prepare a report. 

Send results to those who participated along with a letter of 
thanks. 

Ccmmuaicate results broadly throughout the community. 



1. 
2. 

3. 
4. 
5. 
6. 
7. 

8. 
9. 

10. 
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In conducting a paper and pencil survey by mail several- important facts 
should be kept in mind. 

- ° Survey i nstruments take time ana know-how to develop . Writing 
• directions, statements and questions for a survey takes a good 
deal of effort. Many people will respond and each must have a 
clear understanding of how to respond atf well as what each 
statement and/or question means.. If the survey is unclear in 
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any way, the data gathered may be useless. 

» * 
The respo nse may be limited . Many people do not like to -respond 
to surveys. Only a percentage of those who receive a survey 
instrument will send it back and seme people who do send it back 
will leave some items unanswered.* Account for loss of useful 
survey instruments in your plan and bfe sure to allow time to 
send followup requests and perhaps second copies of the 
instrttaent. * 
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° Compiling , analyzing - and summarizing information is time 

^ consuming. When a survey instrument contains aeverkl items, and 

instruments are received from a large number of people, \ 
• processing thte data can be very time consuming. Allow for \ 
adequate time to hand-tabulate or tb f put the data on a 
computer. It also takes a good deal of staff time to analyze 
/ the information and prepare a survey report. 

° Scheie the survey so that people are available to respond . 

This is particularly important if the target audience includes, 
people from public schools. If the survey comes in the sumner, 
it will li£ around until school opens in the fall. There may 
also be other scheduling problems- to. consider, such as 
n exceptionally busy times around holidays and common vacation 

periods. r 

m The main advantages of surveys are that a good deal of information can, be 
1 gathered from 4 large number of people and that information can be 

analyzed to show areas of agreement and disagreement among*key groups qf 
people. Some of the disadvantages are that constructing. the instrument 
is difficult and response rate may be lo*. Processing the 'data is time 
consuming^anji people may respond in a way they t;hink will be acceptable 
±0 others rather than in the way 4:hey actually feel. V* 

DEUPHI TECHNIQUE 

■ The DELPHI technique is Useful in gaining group consensus without 
face-to-face dialogue. It is a three-round survey that begins with a 
•aries of statements and asks for opinions of individuals about each 
statement. Thfe results of this |irst round are summarized and fed back 
to each participant. Round Two asks individuals to reconsider their 
first responses after seeing the results of the group and possible 
* modifications of the statements. Again, the results are summarized and 
returned to the participants. The third round once more asks individuals 
to reconsider their responses. The technique has been proven successful 
in gaining group consensus. The major benefit is that consensus can be 
■ - 
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reached efficiently* The major problems. of face-to-face interaction art 
overcome* The major disadvantage is that collecting and analyzing the 
, data can be time consuming and costly. % r 



Steps in the DELPHI Process 

1. Design initial questionnaire. 

2. Select group participants. * , 
3., Conduct Round One survey. 

4. Analyze results; feed back to each individual the responses-of 



the group and their own responses. [) 



5. Ask group memb£*£ to reconsifte^ their first responses (Round Two 
survey). / 0 

1 .> 
,6. Analyze the results and feedback to each individual the 

responses of the group and their own fesponses. * ^ 

7. Ask group members to reconsid er their rpspflngg g (p^uti^ Thrift 

survey). 

8. A strong 1 consensus should have been reached. ■ 

s * 

The advantages and disadvantages of the DELPHI technique are similar to 
those for other surveys. The major differences is that/ through the 
three-round process, consensus ds^reSached on the statements in the * 
survey. ' ,f This 4s a powerful advantage. . ♦ ^ 

While the DELPHI technique has many positive features; a few captions are 
in order. « - ■ - % . \^ * 

° Designing DELPHI instruments is not simple . The items must be 
written as statement s* that describe situations / an£ people may 
disagree widely on these statements. Oneysuch statement might 
be "individuals who are considered good employee st^ccme to work 
on time every day." Another example is "Good workers do more 
than their share of the work." * x 

It is also necessary for the statements to be parallel in 
structure and to use language that .will be understood and 
interpreted the same by all respondents. ^ 

Finally, the instrument should include one or more items for ^ 
every area where there is potential conflict. If some areas are 
omitted, the firtal set of competencies and standards may not Be 
acceptable to the participants. 

It is essential to p ^l q^test the instruments with a few people 
* to be sure directions are clear and that the statements have a r 
common meaning. ' 
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° Selecting the participants . Choose people who, represent various 
viewpoints on employ ability development* Have equal v 
representation so as not to bias the results* Be sure that feadh 
individual agrees to participate in all three rounds— drop-otitfe 
will affect the end results* Once the DELPHI is started, keep 
after each person until 'he or she responds. This will take, some 
time/ effort and diplomacy,. 

° Analysis and ^feedback jnustHbe done quickly . Prepare the revised 
instrument, stating numbers of participants who responded each 
~ way and showing changes, and get it back out 'soon.- Keep the 
people involved once the process starts. Lengthy delays wili 
\ , alow people to fprget *bout- their agreements and their views; 

Q-SORT TECHNIQUE ' 

w 

The Q-Sort technique is a type of survey that people respond to by , 
manipulating 1 a deck off cards, with each card containihg a statement. In 
the case of developing employability competencies, the statements would 
be specific competencies. The sorting of the deck is done in order to 
place the statements in stacks representing various degrees of 
importance. Subsequently, respondents can sort within each stack to rank 
order the competencies. The deck or decfc^ can be sorted several times in 
order to get respondents to assess relative importance of various 
coDs>etency statements. After the sorting is complete, the respondent" 
indicates on a response sheet which cards are, in each stack and rarik 
order of cards within a stack. This ntekes the cards* reusable. 

The Q-Sort can be sent to participarfts vi^maj^ or conducted in a group 
setting. The procedure for conducting a survey using the? Q-Sort .method 
follows: 

1. Determine the list of competencies to be included on the cards,. 

* * t * * 

2. Prepare a master^set of. cards,* with one competency statement on 
each car$, and a response sheet. ■ 

3. Duplicate enough sets of cards to conduct the sort process with 
the number of people in the sample. 

4. Identify the people o to be included in the survey. 

5. Ask each person, if he/she will con^lete the survey. 
6: Send the card decks, directions and response sheet. 

7. Collect the response sheets and card decks if the decks are to 
be reused. 

8. Organize, compile, analyze and summarize the data. 

9. Prepare a report and send it ta all participants. 

10. Communicate the results to interested groups and the community 



at large. 
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In conducting a survey usin£ Q-Sort, it is important that the directions 
for sorting and, for recording responses be single and clear. 
Misunderstanding of either will result in unusable information. The 
Q-Sort has the* advantage of interest N in manipulating the ca)rds and the 
novelty of - the approach. It^ also allows people to fcort*4nd resort until 
ttfiey-reach-their^f inal opinions. The disadvantages are* that the cards 
-are costly to duplicate and -it takes a good deal of time to sort a deck 
containing more them a few cards. It., also may be difficult for people to 
discriminate between various competencies in-order to rank them. The 
difference in importance between, two competencies may not be great enough 
for a person to rank one higher than the other. 

The data collection techniques should be selected to reflect local 
conditions* Combinations and modifications xgay be appropriate. The main 
concern is that objectives for the competency^ development effort be set 
and that the techniques chosen result in accomplishment of the objectives 
set. : ' \ - 



' V. FORMAT FOR THE FINAL* PRODUCT 

To be effective , the completed set of competencies must be easy to read 
and understand. It must be ^formatted in a way that allows 4-t *to fit 
easily witfc other program documents such as the Employability Plan and 
Record (EPR>. The final copy must also have an interesting visyial 
quality so that people want *to pick it up and lpok through' it. A good 
deal of the inqpact will be lost, if the qualityjis such that people do not 
use the competencies. Givirrg the final product appropriate quality 
involves decisions about levels of detail, style of expression, visual 
quality and technical accuracy. 

Levels of Detail * x - 

School employees have been writing goals and objectives for stu^ej&t 
learning for decades. The level of detail runs from very gene£»i global 
goals to extremely specific behavior'ally" stated objectives." For example, 
the goals of education for one school district include broad statements 
such as the following: ^ > # 

o Students will gain the skills f op~contipuous learning including 
reading, writing and computing. « . , 



o Students wiUL gain th% skills needed to contribute to society as 

an economically independent citizen. * ^ 

• 

At the other end of the spectrum are behaviorally stated objectives such 
as the following f** 

* 

o Given parts for a sma^l engine, wrenches and other necessary 
tools, and an assembly drawing of the engine, the student will 
assemble thp engine within' 15 minutes with 100 percent accuracy. 
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o Given a list of 15 job titles and a list of three occupational 
^ fields, the youth will match the job titles to occupational fields 

with at least 80 percent accuracy. 

The general goals cover broad fields and can be interpreted differently 
by various people. The behaviorally stated objectives cover specific 
tasks and will probably convey the same meaning tb everyone who reads 
them. The issue is communication, and the, level of abstraction and 
detail chosen must communicate well to arrange of audiences and have as 
little bulk as possible. Writing competency statements is no different 
than writing objectives. The levels of abstraction and detail chosen 
must communiGate the intent' of the program in clear* easy to understand 
langu&ge. ^ 

The style of ( expression used in writing goals, objectives or competencies 
ranges .from outline form to elaborate individual statements. A style 
should be developed that is comfortable for" staff to write and that 
communicates to the various^ audiences that will read arid use the 
competencies. 



One Format: An Example 

While the 1 format — the level of detail and style of expression—should be 
developed. locally, ideas for the format should be collected from other 
sources to serve as a starting point. One such example fallows. It 
contains three levels of detail and is written in short,, easy-to- 
understand statements. The three levels are competencies , indicators and 
benchmarks . The first two levels^ competencies and competency 
indicators, tell what general boAAors you are Peeking; the third level, 
benchmarks, states how well you JK!^ youth to perform thos^- behaviors. 



Competencies 

Competencies are general behaviors that participants are expected 'to 
demonstrate. Competencies result from the question, "What are the m$in 
skills, knowledge and/ attitudes a person should* have if he or she is to 
be considered a valuable employee?" Competencies should be' clear, brief 
and simple; for example: 

\ 

Participants will: 

i 

o Demonstrate dependability * 
o Demonstrate appropriate work attitudes 
' o Describe their personal characteristics that relate to work 
o Describe job market conditions 
o Solve on-the-job problems 
O' Use common tools and equipment 
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Competency Indicators 

Constancy indicators define the specific behaviors that demonstrate 
development of a competency. These behaviors become the basis £br\T ~ 

setting standards and assessing achievement of jccn^e^eilcy development. i# 

Competency indicators are the result of asking the question, "What 
behaviors ykll indicate that a person has attained competence? " ,The 
competency indicators should include all of the important behavidrs 
^ needed to judge a person competent. A few compentency indicators! follow. 



Participants will: 
Competency 

Demonstrate appropriate 
work attitudes by: . 



Indicators ; 

— Showing interest in the job 

— Observing regulations, policies and 
procedures 

— Caring for the property of others and 
"th e "employ e f 



— Using 

— Doing 



correct communication channels, 
an appropriate share of work 



y£xplain conditions 
associated with work by: 



v 



— Showing a -willingness to leairn 



Describing general characteristics 
of jobs (routine nature of work, need - 
for on-the-job teamwork, neea for 
productivity and quality contfrol, 
relationship between work and leisure, 
etc.) 

— Describing work environments (outside, 
inside, office, factory^ physical 
surroundings, psychological atmosphere, 
etc. ) , i 

Describing the relationship l>etween 
education and experience, ana job entry 
and advancement \ 



Benchmarks are the detailed^ standards of jferf ormance set on competency > 
indicators. Benchmarks must include all of the important standards 
associated- with each competency indicator. Benchmarks result from the 
question, "How well «ust a person perform each aspect of a coo^etency 
indicator to be considered competent?" It is possible for an eit5>loy ability 
program -to set bejnchmajks. This would result in bfe/ichmarks that are 
generally applicable to many employers but not specifically applicable to 
any one employer. It is also possible to allow individual enployers to set 
their own benchmarks. -This would result in many different sets of 
benchmarks, each applying' to a specific employer, but none that are 



Benchmarks (or St a 
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generally useful* A middle* ground is also possible. Employers could agree 
on some standards that apply generally* Standards for a broad occupational 
cluster might be possible* 

A few examples of benchmarks follow: 



Competency 

Demonstrate . 
' interpersonal 
skill bys> 



Indicators 

Describing appropriate 
work' attitudes Ijo the 
following standards: 



Benchmarks 

Talks about the job in 
positive ways 3 



2 1 



7 



Talks about rul^s, policies 7 
and procedures and ,why they / 
are important 3 2/3 

'. * " /• ' 

Talks about what a fair ' . 
share *pf wbfk is and why 
each person must do a fair f 
share 3 2 3 



Scale : 



3 
2 
1 



Asks questions that lead 
% to new learning on the 

% / - job * 3 

nearly/ every day, but at least several times a week * .* 



2 1. 




occasionally, once or twice a week 
almoaf: never, less than once a week 

/ I \ ' * 

If a set of competencies is! well developed/ it is pq/ssible to show as 
much or/ as little i^ifoxxoation to/pirticipants and the community as 
necessary.- A list of competencies alone should shows the overall intent 
of the/ program and/not be /a bulky document. If en^lpyers want to know 
what t£he program is doing/abdpt work attitudes, for example, it is * 

ssible to pull but a few competencies, indicators and benchmarks to 
.show/the employe*. Such/a set of competencies has many uses and can take 
many forms. In /general/ however, it provides a- roadmap for the program. 



ts shqiild bevdeveloped locally, but^Ldeas from which to 
J Appendix C contains sample starter lists of 
fc^ir areas. There, is no claim that the san$>le lists are 
Jalre only examples. 

The fi^ts include competencies and competency indicators'. Benchmarks 
(standards) are not included as the lists* would become too bulky and 
benchmarks should be developed locally. - * 

/ v 
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Visual Quality' and Technical Accuracy * 

If the constancy set is tot be widely districted and used/ it is very 
inqportani that the visual quality be inviting. People teftd to pick up , 
and read printed materials that are visually appealing. Visual appeal 
can be attained through good layout, use of illustrations, bald headings* 
an^iw>re^^i/on^type a£tyle. / 

Technical accuracyhas to fco^with correct grarim&r, spelling and typing. 
Using an editor for thi^ purpose is important and not expensive., 
Mistakes in documents that are circulated widely cast doubt on the * 
program's" ability to deliver quality developmental 'experiences to youths 
Make the final product right., ■ 



VI. • JUDGING THE WORK 



As competencies are developed it becomes necessary to have criteria 
against which to judge their quality. N The following 9 is" an example of. a 
set of criteria. Local programs shoul<| modify the criteria^ to sui^ local 
conditions. *■ \ 



Criteria for Competencies 



1. Does the set of competencies include all of the general ^behaviors, 
that yototh are expected i'to demonstrate? Is the set comprehensive? 

• • % ■* 

2. Is the set of competencies manageable? Can y^outh attain *he 

^ competencies? Can the* set o& competencies be achieved with 

available resources? ■ *} ' V - ✓ ; 

3. c Doep each competency describe an important skill, knowledge* or . 
attitude? Should each competency be included? 

4. Is the general behavior described Clearly?* Will people understand 
what i$ meant by the competency? W|J.L there be consistent , 
understanding? 6 , 

5. Is the competency free x>f wording thkt suggests some sopial, 
occupational or life roles shodld be valued more than others? 

. ~ ^ t 

6. Is the competency applicable across ^lines of sex< socio-economic 

status, racef*ru£ral and urban settings, and religioufe persuasions 

* in your community? 

A « ■ . 

Criteria for Indicators 

1. Does eaqh set of competency, ii^icators iijcl'ude all of the 

specific behayiors that youth must -demonstrate in order to show 
achievement of the competency? Is the set conqprehensive? 

: ' v • ■ ' ' ' ' ■ 



29 

■ 40 



2. Can youth attain the competency indicators? 

3. Does each competency indicator represent ai} .important skill , 
knowledge^r attitude? Should each be included? 

Is each competency indicator expressed clearly? Will people, * 
understand what is meant? Will the understanding be consistent? 

Is it possible to set benchmarks for the competency indicator^ 
Can an assessment approach and measurement device be found or 
developed? 



4. 
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Criteria for Standards (Benchmarks) 

!• is there a standard for each indicator? . 

2. * Is each standard expressed clearly so that it is interpreted 

consistently' by program paticipants, employers and others? 

3. Is each standard acceptable to employers? 

*-$ ! 

4. Does each standard set a level K of perfprmance that is attainable 
i>y participants? 

5. Is the achievement of each standard measurable? 



VII. 



1*ELLJNG TOE, STORY 



Developing a good set of competencies is rewarding. To have a complete, 
well-formatted, technically correct and visually" appealing set t>f 
competencies* is good". But it is only the beginning. 1* is essential 
that the competency set be used to communicate the intent of 
employability. development programs to key individuals and groups. Also, 
be sure to 'let people know how competencies are working after they have 
been employed over specific periods of time. Tell the story! 



Who^to Tell 

There arp a number of potential audiences for the story. The specific 
groups to tell should be chosen locally, but consideration shpuld lie 
given to the fallowing groups:* 

. o Staff from the local employability program . 4 J 

o Staff f rom non-public ♦alternative schpols, and other agencies 
a that provide employability related programs and experiences 

o Staff from public schools 

o Employers in both public and private fetors 

o Individuals who receive employability services 

o 1 Parents of youth who receive employability services 



ERLC 



30 



'41 



o Elected officials at local/ state and national levels 
o * Local/ state and federal administrators 



Communication Techniques 

Volumes have been written on techniques for communicating with various 
target audiences. This sectioa will briefly describe a few of the 
techniques- most commonly used to tell the story ateut enployability 
development programs. 



Targeted mailings. 



News articles.^ 



> ' 0 
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Radio and television stories . 



Personal contacts. 



Sending a well-written letter to key 
individuals or groups is an effective 
technique, particularly if it is 
personalized* The letter should 'be 
relatively shorts (one page only) and giye a 
little information about the process used 
to develop the competencies as well as a 
few key, id^as .about the conpetency set. A 
competency set in short form should be 
included as an enclosure. 

Short artiqles describing the conpetency 
development process and the competency set 
is a way to spread the work wiclely. The 
way to get this done is by establishing a 
contact with a reporter or other newspaper 
employee. News articles can get published 
in either of two ways. The first is to 
prepare a news release' and send it to the 
newspaper. The second is to get a reporter 
td interview people and write an. article 
fo3^ publication. Articles prepared by 
reporters a're more likely to be printed 
than are pre-written news releases. ' * 

It may be possible to convince a radio or 
television station to cover employability 
development efforts as a news item or tp 
interview key staff. Most radio and 
television stations have local talk shows 
and empioyability may be an' appropriate 
topic. Public service announcements are 
e^sy to prepare, reach a wid'e audience and 
are free. y 

In some cases it may be good to personally 
deliver a competency set to an individual. 
Thi^type:>of personal attention works well 
with many influential individuals. It 
makes them feel important and it -gives you 
a chance to tell the story first-hand. 
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Presentations . Another good. way to personally give the * 

results of competency .development efforts 
^ is through presentations to key groups. If 

this is done, preparation should be 
thorough so that it is crisp and 
informative. 

Combinations of communication techniques should be used so that all 
target audiences are reached. The specific approach must fit local 
conditions. ■ - 

Telling the story of a competency development process and final product 
should be taken seriously. A plan should be developed that includes the 
following 'sections : 



o 
o 
o 
o 
o 



Objectives of 'the *fcoipmunicationS' effort 
Target audiences to be reached" 

Communication techniques to be used with each audience 
Materials needed to carry out the approach 
Techniques to assess the effect of the effort ^ 



Whei* the plan is complete, move ahe*d by developing the required 
materials, including letters, news releases, short competency sets, * 
descriptive brochures, transparencies and other audiovisual aids. Review 
materials to be used in the communication effort. Quality control must ■ ' 
be strict. When the material^ are ready, the planned activities should 
be .conducted, the impact assessed and adjustment made if the objectives 
have not been achieved. 



VIII. REVIEW AND RENEW 



Competency sets should be reviewed and updated about every three years. 
Conditions change, priorities change and the needs of people change. 
Keeping competency sets current is essential if the program is to remain 
vital. ° 

The^ process *f or updating is the same as for the initial development. The 
big difference is that a set exists to work onk The need is not to 
remake .^e set, but rather to review, confirm ajid/or change the existing 
set. Updating will take much less time and .resources than the initial 
work. But updating is itoportant. 
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Chapter Three: ASSESSING COMPETENCIES 



I • OVERVIEW 



Assessment is conducted to determine a program participant's readiness 
fo£ further training or employment experiences. Every assessment 
activity should be undertaken to help reach a. decision (% : g*, "Can the- 
participant read at a ninth-grade level? *) 0 and if it is not clear what . 
decision is*being made, then assessment should probably not be done.- The 
morfe assessment is focused on a clearly defined competency, the easier it 
will.be to relate results directly to progress in achieving 'the desired' 
competency.* s 4 4 % " 



Paul is learning to prepare a resume that includes a list of ' skills he 
has. An assessment method that is closely linked to this conqp^tency 
would be for Paul to -prepare a sample resume. 



It is important to ketep in mind that no assessment instrument g£ves 
perfectly accurate information on the level and nature of >an individual's, 
competency. *This is particularly true for 6 individuals whose competency 
level is borderline. - For example, an employer may set a cutoff score on *' 
a clerical skills test. However, if xthis stfore is vrigidly adhered tc 
(i.e., any person falling below' it i,s not considered for the job), si 
competent candidates may be eliminated and' some incompetent ones hired 
who fell just below or above the cutoff score, respectively. If the 
employer gave another form of clerical test to those examinees who fell 
within a f'ew points of the cutoff, he or she might well find that some, 
examinees wouldJbe classified differently on the second te'st. Thus, 
results of any assessment should be viewed with a healthy skepticism. 
Wherever possible, assessment data should be supplemented with 'other 
kinds of available information. 



One*further caveat: the assessment approaches discussed here are geared, 
toward obtaining information , on individual slather than groups because we* 
are interested in learning the status and progress of individuals irf- 
reaching their individual competency goals. Group information is needed 
for program level planning and evaluation, and procedures for these 
activities are discussed in Chapter Four (Managing A Competency-Based 
Program) % >\/ ' 



♦The assessment techniques dealt with in this chapter are tied directly 
to pre-employment,, work malturity, education and occupation competencies. 
Assessment of general aptitudes or interests may help to form an initial 
training plan/ but instruments of this type are not a major focus of this 
chapter. 
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II. PURPOSES FOR ASSESSMENT 



In\mployment and training programs, assessment of competencies, falls in 
one of three categories, each- intended to serve a different purpose: , 

• o Initial planning assessment is designed to help determine an 
incoming trainee's level of compet ency so that appropriate 
training or services can be planned. 

o Assessment for management of instruction and training is used by * 
instructors as they make frequent decisions about the next ^ 
activity trainees should engage in to increase tljeir proficiency 
on the competency. 

9 

O Assessment for external reporting , (most frequently to employers) 
is used to make decisions about the readiness .of a potential \ 
• employee for a given job. . , 

It is crucial to know the purpose of the assessment before it is done, 
since the procedures and instruments are likely to vary depending on 
which type of decision is to be servesL— - 

Instruments used for initial assessments usually need to cover several 
competencies . with little detailed information being provided^ each. 
For example, a general reading. test may be sufficient to tell a program*, 
administrator that the examinee does or does not need help with reading 
skills. 5be administrator may not need detailed information on specific 
reading skills (e.g., identifying main ideas, drawing inferences, 
vocabulary). However, the instructor (for instructional/training 
decisions) may need a test which does jive information on whether the* ^ 
examinee can identify main ideas or knows certain vocabulary. 

Assessment ■ for external reporting heeds to be' clearly oriented toward the 
^decision jto_be_made^ , For example, if an employer states that reading 
skills are needed on the jobr it is of little use to receive a score on a 
standardized reading test.* What is needed is information on whether the 
potential employee can perform the reading tasks required on that job. 
Therefore, the assessment instrument needs to be based on reading tasks 
relevant to the specific jbb under consideration. Since these are very 
serious decisions (employment or not), the test needs to be 
well-developed to ensure that it is both -reliable and valid •* 



*A reliable instrument is one which gives consistent information from 
one time to another or from one instrument to another designed to 
measure the same competency. A valid instrument is one which 
measures precisely the- competency it is designed to measure. , 



To summarize: 



Purpose 


Charac t e ri sti cs 


Initial asssessment * 


o 


Used by 'program .administrators 




o 


Covers broad competencies or key 
competencies which will assist in 
training or instructional placement . 




o 


Will vary depending on characteristics of 
the training/instructional options (needs 
to be matched to available options) 


Instructional/Training 
Decisipns 


o 

s 


■m 

Used by instructors/trainers to make ' 
frequent decisions ^bout the next 
instructional activity needed 


* 


o 


Focuses on competencies at a more 
detail ecf level than needed for initial 
assessment * 


# 


o 


May be -less rigorously developed since 
the decisions made could be ^quickly 
modified if incorrect 


^eternal Reporting 

i ? 


o 


% 

Used by employers" or their agents to 
determine examinee's appropriateness for 
a job ■ * 




a. 


Must be clearly tied to relevant job tasks 




o 


Must be technically well-developed 
because of the seriousness of the 
consequences of an inaccurate tdecision 


' Figure 3-1: 


Summary of Assessment Purposes » 
and Characteristics 

\ 


III. 


ASSESSMENT METHODS 



Numerous assessment approaches are available, but there are only a few- 
essential differences among them. These differences revolve around the 
three basic components of air/ assessment: stimulus/ responsfe and scoring 
aethod. In all assessment methods/ some stimulus is given to the 
examinee (e.$., a paper with questions on it. or parts of a machine 
needing 'to' be assembled) which requires some response (e.g./ writing the 
answers* to questions or assembling the' machine) which wilL be scored 
according to jaome established criteria (e. g. # answers given are compared 



to a list of previously established correct answers, or the machine's 
smooth and efficient operation is judged)* 

Various combinations of stimulus/ response and scoring methods have 
evolved over time for measuring competence. Three basic types of 
assessment methods exist: (1) behavioral observation; (2X~product 
reviews; (3) oral or written questidns. 

The major factors which determine the appropriate method to select are: 

(1) the nature of the con^etency (a skill, knowledge or attitude); and 

(2) whether the competency involves a process people use 6r a product 
they produce. Each assessment approach discussed below highlights these 
factors. r 



Behavioral Observation s ^ 

When a competency .focuses on the application of a skin, knowledge or 
attitude, the most appropriate type of assessment is frequently 
behavioral observation. This is particularly true if the competency 
relates to a process outcome. For example, many speaking and listening 
skills can only be^ assessed accurately by observing a person in a 
situation requiring^demonstration of the skill. Asking people if they 
listen cSrefully whesn the boss is giving instructions is less accurate a 
measure of their listening skills than observing them in a situation 
requiring listening. 

Two general situations ..for behavioral observation exist: (1) on the job 

and (2) in simulated situations. Certain x:ompetencies, e.g. , 

punctuality, can bestjpe observed on the job, f Others, such as 
participation in group discussion, can be observed in a simulated 
situation. The major advantage of on-the-job observation is that it, has 
greater fidelity to real life. Its major -disadvantage is that it dpl^ 
not allow the amount of control of the situation that may be necessary to 
assess competencies accurately • Of cotirse in some situations such 
^control is unne cessary or undesirable, as in assessing "punctuali ty* 
Simulated situations .have just the opposite major advantage anc 
disadvantage: th^y *re not as true to real' life' but they allow greater 
control of the situation, thus improving the assessment's accuracy. Both 
types have the .disadvantage of usually being expensive to develop and 
conduct • 



Product review 

In many cases, the presence of skills cam only be Remonstrated by having 
the person produce a product, e.g., write a report, reassemble a 
carburetor, build a bookshelf. The assessment then entails use of a 
rating form or checklist to determine the quality of the product. 
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While a paper and pencil test may show whether, a persoft c has prerequisite 
knowledge, the product review shows whether the person citf actually 
produce a product. For "example, someone may show knowledge of correct * 
gelling, punctuation and (> sentence structure via a multiple choice t*st. 
But not until the person actually writes (for exangple) export which is 
reviewed against established criteria do we have confidence that he or 
she has adequate writing skills for a given job. 

The disadvantage of the product review assessment method is that man/ 
products are very expensive or- time consuming to produce and, of course, 
many important competencies do not lend themselves to this approach. 



Oral and Written Questions 

The two basic types of oral and written question formats are & 
(1). open-ended questions where the respondent generates an original 
answer and (2) multiple choice questions where the respondent selects 
from among two or more possible answers. When the" person whose 
competency is being measured has limited reading skills and reading is 
not the skill being measured; questions can generally be given and 
responded to orally. , > , 

Oral or written questions are best used for assessing whether people have 
acquired knowledge or can apply knowledge by mentally solving a problem. 
Questions can also be used to give* an indica€ion of attitudes and 
interests, but such assessment results must be -very, carefully * 
\ int V^ x ^ ete ^!LiJ^i ± ^^ eB and interests can fluctuate , markedly from day to 
day^6r"sit^tioh"td situation. And written or oral questions of attitude 
and interest are not always accurately responded to because people tend 
to give socially acceptable answers. 

._T??e_ major advantages of openrended questions are that' (1) they "are 
'usually easier to develop than multiple choice ones, -(2) they allow for 

people to give a full rknge' of responses, and (3) they are more efficient 
~*frd-^ing3q?ensire-tto o x^product review. ^The 

disadvantages are that (1) they require more response time, (2) people 
are often not motivated to give the extent of response desired, (3) 
answers are time consuming to analyze, and j(4^they require mote 
expertise on the part of the analyzer than would be true for multiple 
choice questions. 

*" * * 

The main advantages of multiple choice questions are that thpy m (l) can be 
used to test; knowledge across many levels of complexity, (2) can be used 
to measure several different competencies w^ifcfcp one instrument, (3) are 
efficient, (4) are easy to score, (5) can be scored by nonprofessionals 
or by machine* and (6) can provide valuable diagnostic information'. The 
main disadvantage of multiple choice' questions is that examinees with low 
reading ability or limited Experience taking tests will encounter 
problems answering them. In such cases., scores would 
reflect a lack of adequate reading skill rather than a degree of 
knowledge. Multiple choice questions do Require a 'person to chooser from 
among a limited set of ^responses rather than creating their own. In real 
life such restricted options are often not given. Multiple choice 
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questions also nay be carelessly responded to and, in the case of 
attitude measures, are likely to be responded to according to what is 
socially accepted. 

Figure 3-2 summarizes thq characteristics of the vaj^Sus assessment 
methods and Figure 3-3 gives examples of the types of competencies tJjat: 
are^likely to be ^assessed by each method. The example competencies are 
drawn from the sample starter *list in Appendix C. 
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IV, DEVELOPING ASSESSMENT INSTRUMENTS 




Descriptions and Guidelines f par Development 



Four basic types of measurement instruments can be used to gather data 
(tAie term "measurement .instruments" i S used to mean the form for 
recording responses of the person being assessed): 

o Checklists 

o Rating Scales aL 

o Open-finded Question^ 

o Multiple Choice Questions. 

Checklists and rating scales are used for behavioral observation and 
product review assessment * SPhe checklist or rating; scale directs the 
observer or product reviewer to the aspects of the situation or prbdubt 
to which they should' be attending. 

• * 

Open-ended and multiple choice^Sets of questions can lifoftqgs ented to^£hq 
respondent orally or in writing . Responses may peTgiwF orally or in 
writing. The choice between? oral and written depends ^n the skills or 
knowledge being assessed and the oral and written skills of the 
resporident. " For instance, consider an instrument iwitially designed to 
be given in written language. If the skill to fie demonstrated -were the 

, ability to read, the instrument, *would not be given orally. But if the 
purpose were' to ass^ss^pomputational skills* the assessment could usually 
be done orally without invalidating the assessment* The following pages 
describe each £>f " these methoids and provide guidelines for their 

" d^velopmen^^^ , » «*■ 

0 V ^ ^ ■ 

V Before* preparing- checklist* and rating scales/ one should think through 
the structure of , the *j±uat'£bn in j*rhich they will be used. The questions 
; in Pigu5:e|j3-4 itelow wiS-a^ist ff in ;thi*s step. 4 . 



V 

6. > 
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Figure 3-4: BEHAVIORAL OBSERVATION AND PRODUCT REVIEW DEVELOPMENT GUIDE 



Stimulus Description 



1. What form will, the situation take?'' 

a. simulated situation 

b. % on-the-job situation 



2. Wh£t specific tasks will the examinee be called upon to do, and 
what will the instructions be? 



3. How will the tasks be described to the examinee? 

a. t no description beyond that which is regularly 

provided as part of the actual work environment 

b.\ written description 

c. o„ral description 

d. ' other (specify) ; 



4. Will the criteria of acceptable performance be, made explicit as 
part of the exercise? \ 

L . a# y es 

| b. no 

Response ^'Description ' ' , " * «• 

I. 1 Whit is t^o be evaluated? 

' ^ a. a product • 

, ; b. a process 

•{ , '. * <*> 

2* How much evic&nce 'of proficiency will be gathered? 
a. one exercise, one time 
, >b» multiple exercises, one time , 

c. multiple exercises, over a period of time 

• f 

3. Will- the exeuninee.be aware that t;he evaluation is taking place? 
_____ a. yes 

_ ___ b.'- no < 

4. Describe the actual nature of the desired response. 
& Scoring Description - . / 

1. What will be rated? 

a. overall performance 
» b. individual components of* performance 

2. State the specific scoring criteria,* 



- northwest services council 

O* BOX 2009) BELUNGHAM, WA 98227 

(206) 396-1828 



ONE HUNDRED FIFTEEN WEST MAGNOLIA (P.O. BOX 2009) BELUNGHAM, WA 98227 (206) 67M&0 ' ' 



• v EMPLOYEE PROGRESS REPORT ■ 
PARTICIPANT ; JOB TITLE ^ 

HOST 'AGENCY DAT E _ ; v • -J 

* *• 

_ _ ; _ WORK MATURITY " * ~ " r " 

— > — ; — i r 

The Northwest Services Council !, s assessment system is based on a participant^ success- 
ful demonstration of basic and advanced competencies. In order to document the progress 
of youth toward achieving specific competency levels, the folftwing rating tool is a way 
in which a participant's competencies in the skill area of Work Maturity "'fan be. measured 
and communicated • ^ 

Work Maturity is defined as the ability to apply skills on the job in a manner that 
meets employer expectations with regard to basic employee behaviors and responsibilities. 
Work Maturity skills are broken down into ten basic "Competency Categories'!: a statement 
describing the competency. Each Competency Category has several "indicators" f or descrip- 
tors, which are activities or behaviors whi£h are considered critical to the Work Maturity 
skill area. ^ 

INSTRUCTIONS F OR COMPLETING . ' 

\ 

9 • 

A. Read the Competency Category Statement and examine each indicator listed below the 
definition. . ^N^^ 

B. Jt^.Ajfe termination qff*whe£her the participant is satisfactorily performing that 

indicator, v or needs to improve and enter a check (v^ in the appropriate space, 

C. Rate each' Competency Category Statement, based on all of the indicators, using 

the rating scale below. Enter a 1-6 in the rating box for each Competency Category. 

RATING- INTERPRETATION ' 



!T~^"" ~Superior Proficiency - Demonstrates above standard proficiency and a very high degree 

of consistency of performance. 
5 - Satisfactory Proficiency - Demonstrates satisfactory proficiency and consistency on 

all indicators, , 

4 • Approaching Satis factory "Proficiency - Has mastery over mout indicators and is 

r^/^ ■ attempting to achieve higher consistency 

^ of performance. * 

3 - Partial Proficiency - Has mastery of some indicators /tasks associated with this 

competency. ?• , 

2 - Marginal Proficiency - Participant beginning to learn the basic tasks associated with 
— this competency or demonstrates elementary proficiency* from 

prior, experiences. * «^ 

1 - Nq Proficiency - Is entering activity with no relevant prior experience associated 
! with this competency £r demonstrates little progres in developing 
proficiency. 

i ♦ '* * >. 

Ada any comments you may have which could provide additional information participant 
performance of. these competencies. 
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EMPLOYEE PROGRESS REPORT . 
Page 2 

RELIABILITY: Shows responsibility and dependability in carrying out tasks. 

s 

Satisfactory Needs Improvement 

1. On the job regularly 

2. Is prompt* 

< 

3. s Notifies-ifHtate/absent:- 

4. Completes tasks on tine. 



5. Volunteers extra effort whe^ . * j ^ 



necessary . 



RATING 

1 

I 



SELF-MANAGEMENT : Maintain J personal life iri order to function well at work, sr 



1. Maintains physical, mental health. 

/ * ' . . 

2. Is not tired, drunk or/stoned. 

3» Is clean and neat. . / ^ 
4, Has adequate transportation. 
5« tias satisfactory housing. 



6. Settles personal problems outside > ^ RATING 
work. / . / ' • * 



/ 7,. Has adequate child care. ■ * 

' \ ■•/ . 

PERSONNEL FUNCTIONS* Shows understanding of the employer's personnel policies, 

I practices and procedures. 



I 



IT. tjnderstands/ employment conditions. 

2. Understands work standards and 
scti^dulei. 



3. Understands personnel procedures fc 

4. Relates appropriately to supervisor tf 

5. Keeps work records. / ] ' '> RATING 

6. Keeps to dress and safety requirements. * ? 

SUPERVISION: .Accepts guidance, criticism and assignments from supervisor. ^ 

1. Follows instructions. * 



2. Requests clarification when needed. RATING 

3. Negotiates differences of' opinion. 

- - * ~ : " 

J . ' » 



\ 



; p« g « 3- v 

t - 

SUPERVISION - Continued 

4. Participates in training. 

5. Accepts guidance /correct ion. 

, , 



Satisfactory 



Heeds 



ovement 



6 # Willing to accept additional tasks, 



TASK MANAGEMENT : Plans and organizes job responsibilities. 

1. Prepares /plans work, * ' 

2. Sets up materials /tools. 

a 

3. Works consistently to complete tasks. 

4. Uses appropriate techniques/tasks .° 4 > 

5. Uses time wisely. ' , 

*6. Cleans up. 



PROBLEM SOLVING : *S61ves routine and non-routine problems in carrying out job tasks. 

1. Recognizes and analyzes problems. r * / ' 

2. - Doesn't blame others. * ' - \ n 

3. Identifies eaffcly signs of problems. 
_4. Consults with, others,- gathers information. 

5. Considers alternatives. 




6. Changes bad habits; 
MOTIVATION* : 

1.. Is willing to learn. 

2. Does all tasks. 

3. Inquires about job and related work. 

4. Volunteers to learn new skills. 

* * 

. 5. "Seeks opportunities to learn. 

■ / 

o. Practices 'skills/requests feedback. 

T. Seeks additional work when assigned tasks 

coapleted. ■ ' 

- • ' / • • 

— ' /' 

/ 
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COMMUNICATION j Sends clear messages and is able to correctly understand the 
messages of others , * 



Satisfactory 



Needs Improvement 



1/ Uses work terms correctly. 

9 

2. Communicates completely and directly. 

3. Checks for understanding of their own 
communication . 

4. Listens and checks others 1 communication. 



RELATIONSHIPS: Has constructive and proper relationships with other wgrkers. 
1. Is considerate of others. 

• - r. _ 



\ 



2, Is cooperative. 

r 

3* Giy^_helpi_ul_feedMclu_ 



4. Settles problems constructively. 
5.. Is sensitive to others. 



. INFORMATION MANAGEMENT : Recofds, organizes and reports information on the jotr % 

* ■. - 

1. Selects information for recording. . 

2. 'Provides writ ten /verbal information. 

3* Uses forms % correctly. . \ - 

4. Records "information legibly. ; ' ^ / . m 

5. Uses" good grammar /spelling. , 

6. - K/eeps track of records, > _^ v r 

^ * * 

comments , . ■ : ; : 



RATINf 



RATIN' 



RATIN( 



EVALUATOR'S SIGNATURE 



. PARTICIPANT'S. SIGNATURE 

• * " > 
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Open-Ended and Multiple Choice 

a 

The distinctions between multiple* choice and oped-ended questions have 
already been discussed. Hie review questions in Figure 3-6 are intended 
tojyjide the person developing an instrument^ of one of these types. A 
later section in this chapter, entitled "Existing Instruments," presents 
a set of criteria for review of instruments available from commercial 
publishers. 



Figure 3-6: REVIEW CRITERIA FOR WRITTEN OR ORAL QUESTIONS 

Ask yourself these questions when developing any type of written or oral 
questions. The first set applies to any written or oral questions. The 
next set apjplies specifically to multiple choice* questions, and the final 
set applies to open-ended. quest ions. Each question should receive a 
"yes" answer. 



General Criteria 

1. Have I avoided using items that are presented in an ambiguous 
fashion? * J- 

2. Have I followed standard rules of punctuation and grammar in t 
constructing items? j 

*3. Have I constructed only items that have, right or clearly ,best 
answers?* 

t 4. Have' I kept the reading difficulty of test items low for the people 
being tested? 

5. Have I a voided^ constructing the items from statements taken verbatim 
from instructional materials? , 

6. Jf any items are based on an opinion or authority, have I stated 
1 whose opinion or what authority? 

c 

7. Do items avoid offering clues for answering other questions? 

8. DO the items avoi'd irrelevant clues? 

9. ~ Have» I avoided making any items overly difficult by requiring 

unnecessarily exact or difficult operations? 

10. ", Do my items avoid using words such as "always," "never ,*" "none,", 

"all"? ' < 

11. Have I avoided "trick" questions? ' 

12. Have I checked the questions with 4 other knowledgeable people to try 
to eliminate ambiguity, technical errors, and other errors? 



*Applie$ only to knowledge* tests** 
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13* tfo the questions try to measure only a single idea? 

14. Have I restricted the number of" question formats in the measurement 
instrument? ( 

15. Were the mos't "valid" formats used? 

16 Have I grouped questions presented in the same^ f ozma't? . 

17. Do the correct or desired answers follow essentially a random* 
' pattern? 

Criteria for Multiple Choice Questions 

!• Is each question designed to "measure a desired competency? 

2. Does the litem stem* clearly define a problem? * 

3. _ Have, I included as much of the item in the. stem as possible? 

4. Have I avoided irrelevant material in>the item stem? 

> 

5. Have I avoided grammatical cues in the item stem? 

6. Have I kept to a minimum the number of negatively stated item stems? 

7. If the negative is used in an item stem, have I clearly - emphasized 
it? 

8. Is there one correct or clearly best answer? 

-< 

9. " Have I avoided the use of answers such as "all of the above" and 

"none of the above"? 

10. Have I made sure that all answers are grammatically consistent with 
the item stem and parallel in form? * 

r «v . . ^ 

11. Have I avaided~"stlting the correct or desired answer in more detail? 

12. Have I made sure that all distractors*** ^present plausible * 
alternatives to etfiuninees who do not possess the skill measured by 
the item? n - 

13. Have I avoided 'including two answers that meaJTthe same, such^hat 
both can be rejected? * - v 

14. Have I avoided the use of modifiers like "sometimes" and "usually" 
in the alternatives? * 

15. Have I made sure to use important sounding words v in the distaractors 
as well as in the correct or desired answers? 

Ii6. Are-all- answers the same level of complexity? 



* An item stem is the part of the question or statement presented prior 

to the list of alternative answers. 
** Applies only to knowledge tests. ' # 

*** Term used' to refer to incorrect answers* 



17. Have I made the' answers as alike in style as possible? 

18*. Have I varied the length of the correct answer, thereby eliminating 
length potential clue? 

19. Have I ^listed answers on separate lines, beneath each other? - 

20. ^ Do I have sufficient questions (usually at least three) measuring 

each concept, to ensure reliable results? % 

21* Have I used new material for the' students in formulating problems to 
measure understanding or ability to apply principles? 

< \ 
Criteria for vQ^en-Ended Questions 

1. Are these types of questions only being used wheji multiple choic? 
ones are inappropriate? . 

21 Are the questions closely matched to the competencies they wer 
written to measure? 

* 

3. Does each question present a clear task to the student? 

4. 1^ there sufficient time for answering questions? 

5. Are examinees aware of the time limits? 




6. Dq erfhminees know how many points each question is worth? 

7. Is new and/or interesting matteijial used in the question*? 

8. Do questions start with words or phrases such as "Compare, " 

v "Contrast, " "Give the reason for," "Give original examples g£;" 
"Explain how," "Predict what would happen if," "Criticize," etc? 

9* Has an "ideal*" answer to each ..question .been prepared before 
administering the instrument? 

10. Have scoring procedures been established for each question? 

11., Save questions been- avoided which can be answered by a simple "yes" 
or "no"? 
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V. SELECTING EXISTING" ASSESSMENT INSTRUMENTS . 



On the preceding pages, guidelines were presented for developing your own 
assessment instruments* This section focuses on selecting existing 
instruments. It provides guidelines for such selection, starter lists of 
published instruments in the four competency areas, and reference sources 
that may be useful in obtaining further information. 

An existing instrui&ent will generally be less expensixe^han one you 
develop, and certain aspects of the instrument's technical 'quality are 
likely to be higher. Existing instruments are available from publishers, 
or from persons who have developed their own for a specific use but have 
never published thenu While published tests are generally extensively 
reviewed, pilot tested and revised before publishing, they may not have 
la^ tjie characteristic's important for your particular situation. You 
miiat .carefully review either published or unpublished instruments to be 
sure they can provide useful results. 

Guidelines for Selecting Existing Instruments 

Three general categories of criteria should be used when reviewing 
instruments: validity, usability and reliability. 

Validity refers to the extent to which the instrument measures the 
skills, knowledge or attitudes the user intends it to measure. Thi^B is 
the most crucial issue in selecting an instrument. JUst because an 
instrument's title suggests that it measures job knowledge or reading 
skills does pot mean it measures the ones that are important to you for 
your particular purpose. Each question in the instrument should be 
looked at with a qrAical eye. to determine if a person's answer to that 
question is truly going to i^elp you decide if the person has achieve^ a 
competency and standard relevant to your program. For example, if the, 
instrument measures knowledge about jobs th.at are unavailable in your 
area* is that useful to you? You may find instead that the instrument's 
fQrmat and type of questions provide a useful model for you to develop 
your own instrument.* * « 

Usability of ■ an instrument refers*to the extent to which the .instrument 
is suitable for the context in which it will be used. For example, is 
the method of administration feasible In your program? If it requires 
administration with each person it miy be unusable because of limited 
a^aff-. If the scores are not easily interpreted or if the reading level 
ofW questions is too high for your clientele then the instrument is v 
unusable. 0 

Reliability refers to^he extent .to which an instrument measures a trait 
consistently. Mathematical formulas exist for calculating the 
Reliability of multiple choice instruments', with scores ranging' 
theoretically from^O to 1.00. In reality no instrument is perfectly 




Reliable (i.^. having a reliability bf/l.OO). Good tests of knowledge 
will achieve reliabilities i,n ^^.85/ to .98 range. Instruments 
measuring attitudes tend to, f afl Ifti* the .70-. 90 range. Attitudes are by 
their very nature less stable thaiv knowledge and thus their measures are. 
less reliable. • 



With instruments requiring human judgment in the scoring <«*g*, rating 
forms. or open-ended questions), an importfuit form of reliability is the 
consistency among raters. This type of reliability must i?e determined 
locally by having two or more raters conduct a rating and then comparing 
the „ similarity of their ratings. For help with the ways to check 
interrater reliability, consult an educational measurement textbook, 
program evaluator, research specialist, or a local college or university 
professor of measurement in education. 

Figure 3-7 presents a set of criteria for reviewing existing 
instruments. Although this list contains the major criteria for 
instrument review, you may find that other issues are important to you 
and you will want to add them to this list. 
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Figure 3-7: CRITERIA FOR REVIEWING EXISTING INSTRUMENTS 

I. Validity - the extent to which the test measures what the user 
intends it to measure. 

!• Do the questions measure the specific competencies that need to 
be assessed ? I " 
To answer this question, list the competencies to be assessed. 
Then go through the instrument item by item and determine which 
competency, if anyT^each item measures. On a multiple ^choice 
instrument adequate coverage requires at least three items 
measuring a given competency. If the test measures 75 percent 
of your competencies, the coverage may be adequate for initial 
' planning decisions. It would not necessarily be adequate for 
instructional/training management decisions. 

2» Were the questions developed in a systematic and rigorous 

manner so that the content is adequate and bias is minimized? 
, In the manual accompanying the instrument, the publisher should 
present information on how the questions were selected for 
inclusion. They should be based on a careful analysis of the 
content area with clear linking of questions to the competency - 
they meas ure. Th e questions also should have been reviewed for 
race, sex" and cuituralTBiasT^ 

3 » Were any empirical procedures used for screening or selecting 
items to ensure that items measure what they were designed to 
measure, are understandable r contain reasonable answers* and 
are free of ambiguous alternative answers and unnecessarily 
complex language? 

Empirical procedures include item analysis, review by juries of 
experts, review of item difficulties, criterion-group analyses""* 
4 or factor analyses. 

< 

4» Is the validating group representative of the [ population with 
which the test is to, be used? * 

Include the following considerations in the evaluation of 
validating gfoup representativeness: 

(a) Were both males and females included in the validating 
group? > 

(b) Were all major ethnic groups represented in the validating 
group? 



♦The rating system used on this criterion assumes that the user is 
working with a broad cross section of .participants. There may be cases 
where a more narrow and exclusive instrument is desired. This criterion 
would then be modified to fit the population. 
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(c) Was the validating group a nationally representative 

' >/ , »ample in terms of population density characteristics 

JT (e»g^ urban, suburban, rural, *tc), and geographic 
'* representation of the age group for which the test was 
designed? 

(d) Was the sample obtained systematically through cluster, 
. .stratified or random methods? 

(e) Was the validating done less than five years ago? 

II Usability -**the extent to which^a test is suitable for the context 
in which it will be used. 

!• Are the questions at a suitable language level for your program 
participants? 

. y The questions should be reasonably appropriate in difficulty 
and intellectually stimulating regardless of the competency 
being measured. 

2» Are instructions to the administrator clear and complete? 

3» Are instructions to the respondent written in clear and 
understandable terms? 



Is the instrument clearly formatte^? " 

Is there a single way for respondents to record their responses? 

♦ 

6* Does the instrument yield scores that are easily interpreted 
and appropriate tot their intended use? 

The main issue here is likely to be whether the reported scores 
are norm-referenced or criterion-referenced. The introduction 
to the startfer lists of published instruments presents 
• discission on this topic; • '"^'^ J 




Is the amount of time required for assessment appropr 

III. Reliability - the extpnt to which the 'test measures a £rait with 
consistency. . 

«. * % 

!• Is reported reliability for scores'' sufficiently high? , 
One or more^of threfe different types of reliability' toay be 
reported in the technical manual accompanying the instruments- 
„ * alternate form reliability, test-retest reliability, and 

— — -- internal consistency estimates. Look for reliabilities above' 
• 80 for tests of knowledge. Attitude measures should.be at ' 
least .70. ' 

2» Are the scoring procedures clear and complete, thus ensuring 
reliable scores? 

A scoring guide, template stencil, other straightforward 
process or machine scoring should be provided for scoring of 
multiple choice item's. Scoring guides and rigorous training 
procedures should be provided for scoring questions that 
require' human judgment. 
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VI. METHODS OF ORGAN JZJNG THE' ASSESSMENT PROCESS 



How' will you operate your assessment system? Will the 
system be decentralized or centralized? < Will it be 
computerized for administration or recordkeeping or not 
computerized? Will a separate staff conduct assessment or 
will it be done by the instructional or administrative 
staff? Each 'approach has merits and disadvantages. 



Centralized vs. 
Decentralized 
Administration 



Some agencies fin<J that the most effective way to conduct 
assessment is to haveone centralized assessment center. 
This approach has the advantage of standardizing the 
assessment across all programs and usually means that staff 
with more^ssessment expertise can be hifed. Such an 
approach would not work well, however, if the programs it 
serves are very-scattered and communication with the 
assessment center is poor. It; .may also mean that 
assessment can not be tailored to the needs of each 
program. 



The £urpos£ f^r assessment will also influence whether it 
is donK^pentral^y • Initial assessment and assessment for* 
external reporting can probably be done most effectively in 
a centralized setting. " Assessment for instructional, 
management nearly always needs to be done within each 
instructional job training program because the ^instructor 
needs quick turnaround on the assessment results and needs 
to be heavily involved in determining what should be 
assessed. 



Specialized 
Assessment 
Staff 



The 



A 



In nearly all situations it is advantageous to have at 
least cpe staff member with expertise in assessment* 
_tTOhnicalities of good assessment are many. A highly 
qualified person can play a key role in ensuring that 
appropriate instruments are used and that results are 
interpreted appropriately. On the other hand, it is 
equally critical that the assessment staff work closely 
with instructors and trainers to insure that appropriate' 
content is. being .measured and that the characteristics of 
program participants are well understood. Specifically, 
the assessment staff members need to understand what 
assessment approaches may be threatening to participants in 
your program. > 



Computerized 
Assessment 



Computers can be used in a variety of ways* Work is now 
being done by several test publishing companies to develop 
computerized testing systems. Examinees can sit at a 
computer terminal and respond to test questions as they 




> 



appear on the screen. Based on the examinee 1 s response, 
the computer cfcn next present an easier or harder test 
itto. This tailored branching approach to testing can be 
very; efficient in determining a person's^ strengths and 
weaknesses. Computerized systems are also available for 
determining career interests. 

Computer^ can also be used strictly for recordkeeping. 
Paper and pencil tests ittay be given and scopes entered on a 
computer ,f or ready access to instructors and others, 
qomputerized recordkeeping aids compilation of results 
across program participants for program evaluation purposes 

The major disadvantages of computerized systems are that 
(l)they are generally expensive, (2) they require trained^ 
personnel and '(3) the software to perform the desired — 
functions may not be readily available. However, programs 
would likely find it very valuable to investigate the 
possibilities in theix; area. 



4 
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VII. RESOURCES FOR FURTHER ASSESSMENT INFORMATION 

< 

Listed below are documents, people and organizations which may be helpful 
in planning and conducting assessment: , 

1» Beverly A. Anderson, Guide to Adult junctional Literacy Assessment : 
Using Existing Tests , Northwest Regional Educational Laboratory , 
• (300 S.W. Sixth Avenue, Portland, Oregon 97204), 1981. 

This guide focuses on when and how to use existing literacy tests for 
measuring adult literacy skills. Available tests are listed. It is 
designed for adult educators, counselors and administrators of adult 
educational programs* 

r 

2. Beverly A. Anderson, Richard J. Stiggins, and Suzanne Hiscox, 
Guidelines for Selecting Basic Skills and Life Skills Tests , 
Northwest Regional Educational Laboratory, 1980. 

Procedures for identifying appropriate fcests for measuring basic 
skills and life skills are given. It is, designed for teachers. 

3. Nancy J. Bridgeford and Richard J. Stiggins, Consumer's Guide to" 
Writing Assessment . Northwest Regional Educational Laboratory, 1981. 

This guide presents information on # alternative methods of assessing 
writing skills. Descriptions are given of nearly all available 
writing assessment instruments* People across the United States with 
expertise^ in writing assessment are listed. « 

4. O. Buros, (ed.), lhe Eighth Mental Measurement Yearbook , The Gryphon 
Press (220 Montgomery Street; Highland Park, New Jersey 08904), 1978. 

This book is the best reference available for obtaining critiques of 
existing assessment instruments. School counselors, education 
department libraries at universities, or professors of educational 
measurement would likely have a copy. 



5. ' O. Buros, (ed.), Tests in Print , The Gryphon Press (see address 
— abovel,_1978. " 

This book lists all tests in print and^ gives* information on the 
publisher, author, date and* content of each test^ Hstedi^ Critiques 
of tests are not given. ^ 

6. ERIC Document Reproduction Service, P.O. Box 190, Arlington, Virginia 
22210. 

ERIC (Educational Resources Information Center) is an educational 
information data base. Unpublished documents are collected in ERIC, 
along with journal articles. 
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Scarvia B. Anderson, Samuel Ball,, Rieffiard T. Murphy and Associates/ 
Encyclopedia of Educational Evalu^ii /n (San Francisco: Jossey-Baas 
Publishers)/ 1975* 

Schoolfutures, Inc. (2092 East tfiin Street, Hillsboro, Oregon 97213). 

/ t 

Schoolfutures is an educational brokering agency which will, provide 
descriptive information on available products in the areas of (1) 
Assessment, (2) Pre-Vocational/Career Exploration, X3) Career Based 
Basic Skills, (4) Vocational Suppport Programs and i5 r ) Vocational 
Training "Systems . A Systems Catalog 'is available fbr $7 per area 
specified above or $25 for the entire notebook covering all five 
areas. (Price is F.O.B. Hillsboro, Oregoj^T 
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Chapter Four: MANAGING A COMPETENCY -BXSED PROGRAM 



/ # ' - M I# OVERVIEW 



Keeping the system "on track" is the basic purpose for management, is 
the program performing as expected? What 'weak spots exist? Do results 
justify the investment of resources? v Finding, using and reporting 
answers touthas^cjuestions require management. Management practices make 
it possible' for program to be accountable; that is, to let people who 
pa/^for, the program, those who use services provided by the program and 
other interested people know what the program expects to do and to what 
extent the expectations are being realized. 

In a^competencyrbased program, expectations are stated as program or 
managament goals. These goals- identify the employ ability competency 
that participants are to gain and indicate program performance | 
standards. A few examples of program or management goals follow: 

-~o-~ At' leastr 90 percentrof "those "whcr~enter and" remain in the program 

for sixteen (16) weeks or more will demonstrate minimal work 
o maturity competencies, including dependability, appearance, work 
performance, interpersonal skills* and work adjustment. skills. 



At least 90 percent of thosdi who enter and remain in the program 
for six (6) or more weeks wijll demonstrate pre-employment job 
seeking skills, including lobating job openings', applyihg for a 
job and interviewing for a job. 

o Of those whd^enter and remain in the program for at Ic&st one 
year, 50 percent will demonstrate minimal occupation-specific 
competence according to, standards set by either an occupational 
training program or an employer who is providing on-the-job 
training. ^ 

Program or management goals, as foxx can see, pertain to the overall 
program rather than to any individual. However, program goals Cannot be 
set without Having a set of employability competencies that guide the 
development of participants. Bnployability competencies provide not* only * 

t he b asis f^r assessing the progress of e ach participant but also the_ 

basis for assessing the overall performance of the program. 

With this in mind, then, the first step in managing a program is to 
establish employ ability competencies^and benchmarks. The second step is 
to use the competencies and benchmarks to develop program or management 
goals* Third, the goals are. shared with those who helped develop, them 
and those expected to accomplish' them. As a fourth step, performance in 
the accomplishment of goals is checked. Fifth, program performance ia 
reported and finally, adjustments ar^made as needed. 



©2 KJ t 



1. qevelop Conqpetencies and Benchmarks 

2. Develop Program Goals *for Conqpetency and 

• Benchmark Attainment * * 

3« Share Goals 

4. Check Performance in Accomplishment of Goals 

5. Report Program* Performance 

6. Adjust Goals and/or Competencies and 
Benchmarks - 

• * Figure 5-1: Six Key Management Tapks 
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* * . II. DEVELOPING MANAGEMENT 'GOALS— 

THE STANDARDS FOR PROGRAM PERFORMANCE 

While competencies and benchmarks are developed through widespread 
. involvement o£ many groups and individuals, management goals are 

develo]ped through interaction* with fewer but very > important groups, . Q 
^namely, program atjff, service deliverers, program advisory committee and 
^funding agents. The processes include advice and negotiation, 'because 
-resources usually will not, allow a program to* help all participants 
Require all competencies. Therefore, the management goals must reflect 
priorities* for program performance. Will the program emphasize work 
• maturity competencies trather than occupation-specific competencies? Will 
& major goal be to. upgrade the educational skills of the participants? 
Will' the ;pj^igram work toward minimum competence in large numbers of 
participants 6r concentrate on minimum-plus-additiorial competeftce for 
fewer participants? These and other questions must be answered as 

• management goals are set. * 

Seeking advice comes first in the process of setting management goals. 
Open forum discussions with staff members, advisory committee, service 
deliverer^ and funding agents make *a good .starting point. The ptirpose „of 
this process "is to discover the values and biases of key groups ^nd 
individuals and to find areas of agreement and disagreement . It is- also* 
possible to figure oi^t how various people view and interpret rules and 
^9tt^*tions/that'. guide program operation. The "discussions should revolve 
abound possible management goals and the resources needed to achieve 
.them. This effort should result in a tentative set of -management goals. 

* c « . 

Next- comes negotiation . This is the process of finalizing management 

9©als for the program. . The final set -of goals' will guide the; program for 
v ai\^ agreed upon period of times', one year for Example. While advisory 
' ;? taff and service 'deliverers .should -be in agreement with the 

, manaigeient goals, the main negotiation is betwefen the program and the 

funding agent. A see of goals should be < presented to the funding agent. 
. Th^ funder should suggest changes and the differences should be worked 
out. 'The negotiation centers on goals, and resources because the two. must- 
be in agreement. Whence differences have been worked out the goals 
Should be. put into final written form and both parties should sign off. 

The third step in, , managing a competency-based program is to spread the 
. pord — let people know^about the* management goals. It is very important 

xtJi^t program staff and service deliverers -know about the, final set of 
. t management goals "because these * goals form the basis .upon which -both staff 
and service deliverers plan their work. Program or management* goals can 
also become the .basis for,- negotiating performance objectives with staff 
members *and service deliverers. < . , 7 

"Other groups- and individual should' also know about the management 

• objective? • Bnployers, comaunity organizations/ people eligible for 
fc . program services, "and the' community at large should be informed. 

Technique's- for. spreading the word were discussed in .Chapter Two and apply 
- here also. Use a variety of • techniques to^et people Itnow the management 
goals for the program. 
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Checking performance is the fourth step in program management • 
Assessment of competency development provides the information needed. 
That is, collecting and summarizing data from individual participants 
gives overall program performance data* Let's look at an example. 

•4. 
• ► 

At least 90 percent of €hose who enter and remain in ^cr, program for 
sixteen, (16) or more weeks will demonstrate minimal wor* maturity 
competencies, includinc^dependability , appearance, work performance, 
interpersonal skills and work adjustment skills. The technique used 
to assess .these competencies is a checklist completed weekly by 
employers for each participant. The checklist may look something^ 
like the following one: '» 



WORK MATURITY- CHECKLIST 



Week beginnipg_ 
Participant 



Supervisor 



3 2 - 1 

Excellent Satisfactory 
(Almost (Most of 
Always) the Time) 

' *~ . OfWn) 



Needs 
Improvement 
(Not Very * 



COMPETENCY. INDICATOR 



Comes to work 



Comes to work on time 



Lets supervisor: .know when not' 
coming and/or when coming late 



Groomed for the work 



Dressed for the work 



Does enough work 



Does good work 



Gets along with supervisor 6 



Gets alonfr with other employees 



Works cooperatively 



Is interested in the job 



Follows rules and procedures 



Takes care of property of. 
others and employer 



Is willing to learn 

_ r ^ . — —? — ; 



v Resolves conflicts in 
positive ways 



Keeps trying in spite of 
problems* or setbacks 



Figure '5-3: Sample 'Work Maturity Checklist 
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The benchmark (standard) for these "competencies is that over the 
sixteen (16) week period the individual will receive satisfactory or 
excellenfratings for each competency iifaicator at least fourteen (14) 
timaaf To afceck program performance it is* necessary to identify al'lf 
^persons who were enrolled for sixteen (16) weeks and comg(ite ti\e percent 
who diet the benchmark. If the percent is 90 or more, the program 
(management) goal has been reached. If the program goal is" not reached, 
further analysis of the information is needed. Questions such as the 
following should be answered. ' 

o For which competency indicators is the standard not reached? 
o Is the standard set too high? > 

b Wl^t are the reasons for nonattainment? ■» " * : 

\ o v How should the program be modified to improve results? 

Some, of these questions can be answered from the assessment information 
already available and some will require additional data collection. 

With data in on all management objectives, a report of program 
performance should be prepared and' distributed* This'is the fifth 
management step, a .key element in showing accountability. A sample 
report form follows: 



PROGRAM PERFORMANCE REPORT 



This .report shows the operating results for the Your Town Bnploy ability 
Program for Fiscal Year 1981. The report* includes information on each of 
the fifteen (IS) management goals set at the beginning of the*year. 



Management Goal 



Program Performance 



Comments 



o At -ieast 90 percent 
'of those who enter 
and remain in the. 
•program for sixteen* 
(16*) or more weeks 
will demonstrate* 
Mnimal work maturity 
competencies/ includ- 
ing dependability/ 
appearance^ - work per- 
formance** .interperson- 
al sklJyis and work 
adjustment Skills. , 



Ttfe program narrowly 
missed achieving this 
goal I 82 percent of the 
individuals reached the 
benchmark q£ s receiving 
satisfactory or excel- 
lent .ratings on all 
mi/nimym wbrk maturity 
competencies for four- 
teen (14) out of six- 
teen (16) weeks. 
Assessment with a 
standard rating, form . 
was done weekly by 
employers for each 
participant. 



Several people did 
not reach ther" 
benchmark f of* the 
dependability com- 
petency indicator/ 
which requires 
participants to let 
their supervisor 
know when they are 
not coining to work 
or" are cqjtdng to 
work lite. "Follow- 
ing rules 'and 
procedures" was the 
other competency 
indicator that 
lowered t*fie percent 
of* individuals 
reaching the bench-- 
mark. 
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Similar descriptive material should be presented for each management 
goal* S^oh a report or even a shortened version of. the performance 
results should be shared with staff, service deliverers, advisory group, 
funding agency and the community at large* Techniques for communicating 
with various individuals and groups are included in Chapter Two. 

The sixth and final step in the management cycle is adjusting the program 
if performance is not adequate. .Management gofcls, assessment techniques, 
activities and strategies, ind personnel are all examples of conponents 
which may need adjustment. You wilj. need tq, answer questions like 
these: Who will make the adjustments? What resources will be required 
to*make the adjustment? 



When adjustments to any aspect of the program are ccuplete, communicate 
the changes to interested people and begin the management cycle again. 

The management function, when based on employability competencies ^d 
program goals, can maintain credibility "for. the program. " The program is 
.accountable and people know it. When 4 people believe a program to be 
accountable for increasing valued competencies, support is likely to be 
maintained. 



si 
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Chapter Fivd: PLANNING INDIVIDUAL PROGRAMS 



I. OVERVIEW 



The concept of individualized planning typically scares people away. It 
.is often dismissed as too costly and too inefficient to use in 
employ ability development programs. HoweVer, individualized planning 
pulls the various^parts of a competency-based program together. As such, 
it is fundamental.. There are numerous ways to, carry out the steps of 
individualized planning, and nearly every program should be able to find 
an alternative to fit its unique situation. Suggestions for personalized 
learning activities and services are presented in this chapter. 



II. A RATIONALE FOR INDIVIDUALIZED PLANNING 

Individualized ^planning is a process that synthesizes the various parts 
of a competency-based employability skills program. It usually involves 
. the preparation of a document that describes 

, o Results of initial assessment (competencies and benchmarks 
J already attained) . * * 

o 9 Individual goals (competencies, and benchmarks aimed for within a 
specific time frame) 

o Activities that will be undertaken to preach goals ' * 

o Description of when and how assessment will take place J 

o A record of ^nchmark/competyency attainment. ! 

• i 

„ In areas where pompetency-based programming has taken place, a great 
variety of planning documents have, been used. Figure 5-1 below shows an 
example of *n individualized plan. 
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INDIVIDUALIZED PLANNING RECORD 



A. PARTICIPANT BMPLOYABILIT^ GOAL 



B. REQUIRED COMPETENCY LEVELS: 
PRE-EMPLOYMENT SKILLS: 



J 

"WORK MATURITY: 



BASIC EDUCATIONAL SKILLS: 



OCCUPATIONAL SKILLS/ 



C„ CURRENT COMPETENCY LEVELS: 



PRE-EMPLOYMENT SKILLS': ADEQUATE / / IMPROVEMENT NEEDED £ / 



COMPETENCY AWARDED 



ON ASSESSMENT, PACTS / J 

SUGGESTED SERVICES: 



WORK MATURITY: 



ADEQUATE / / IMPROVEMENT NEEDED / 



COMPETENCY AWARDED 



ON ASSESSMENT PACTS £ / * 

SUGGESTED SERVICES: * 



7 



BASIC EDUCATIONAL SKILLS: ADEQUATE / % / IMPROVEMENT NEEDED £ / 

V. 



COMPETENCY AWARDED 



ON ASSESSMENT FACTS £ / 

' SUGGESTED SERVICES: ■ ■ 



OCCUPATIONAL SKILLS: ADEQUATE £ / IMPROVEMENT NEEDED £ / 

COMPETENCY AWARDED t 

; N ' ON ASSESSMENT FACTS £ / 

SUGGESTED SERVICES : 



D. LIST ANY OTHER MENTAL, PHYSICAL, OR OTHER SIGNIFICANT BARRIERS TO 
EMPLOYMENT: 4 : a . ; . 
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While individualized planning requires a strong commitment from program 
staff (e.g., counselors/ instructional staff^u: whoever is assigned the 
task of negotiating the plan with participant^, the commitment is not 
necessarily one of time* It is rather a commitment of personal 
involvement in whether the individual is successful in attaining his or 
her goals. 

In designing an individualized planning process to meet your needs, you 
may wish to consider some of the issues listed below: % 



POTENTIAL PITFALLS 



POSSIBLE ALTERNATIVES 



1. Individualized planning takes more 
time and staff resources than 
our program has. 

/ 



1. Involve participants in the 
planning process. Let them 
develop their own plans; 
staff can then serve a review 
function. 



2. Individualized planning requires 
juggling of more data and 
information than we can keep 
track of '. 



2. Consider using a micro- 
computer to manage data* 



3. Individualized planning means a 
mountain of paperwork. 



Review the materials you us 
an,d determine whether you 
can cut down on paper. 



4. Individualized planning means we 
serve fewer clients.' * * 



4. . It may be necessary to serve 
few^r clients in order tp 
serve them well. No one Vins 
^ if many clients, receive 

services that are not helpful. 



5\ * Individualized planning is time- 
consuming because participants 
have difficulty engaging in it. 



5. -Make it a learning process. 
Good planning skills can 
serve participants the rest" of 
their lives.; 'tie it to career 
planning processes. 



III. STEPS OF^HE PLANNING PROC|SS 



..While the exact number of steps and the details of how they are carried 
out vary, 'ttiere seem to be four fundamental steps in every individualized 
planning process. These are:- 

>* ' 
o Individual assessment and goal setting 

* o Selection of activities to reach goals • 

o Ongoing assessment of goal Attainment 

o Itevision/adjustment of goals 



s 



This -process takes place in a cycle (Figure 5-2)'-: 
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INDIVIDUAL ASSESSMENT AND 
GOAL/COMPETENCY SETTING ' 



' REVISION/ 
ADJUSTMENT 
OF GOALS/ 
COMPETENCIES 



SELECTION 
OF ACTIVITIES 
TO ATTAIN 
COMPETENCIES 



/ 



ONGOING ASSESSMENT 
OF COMPETENCY 
ATTAINMENT 



FIGURE 5-2: CYCLE* OF INDIVIDUALIZED PLANNING 



• ■ /. 

/ 

." / 
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If an individual's plan is to have meaning,- all parts of the 
employability program must work togetKer in a complementary fashion* The 
most useful plans contain well-analyzed, sequenced sets of learning/ / 
activities individually negotiated with participants to move them toward 
clearly defined goals. Plah development is an ongoing process; learning 
and work experiences are structured and goals are refined throughout the 
program cycle. ; 

Ideally, the pl^xx Should be contained 'in a folder that is easily 
accessible to staff and participants ^his gives the participant a 

tangible record of competencies, aimed for a$d attained, and can be 
invaluable in the job-seeking process. \ 

The concept and practice of negotiation is absolutely essential to 
employ aba lity plan development. In a broad sense, the spirit of 
negotiation characterizes the atmosphere within which participants and 
staff continually exchange ideas. 



Giving participants realistic experiences in negotiation is essential to 
preparing them for responsible^roles in the community. As much as f 
possible, participants shouldToe free to decide for themselves what their 
career and life goals are anfi what planning and activities are needed to 
achieve those goals. Negotiation encourages decision making by providing 
for continuous information exchange^among participants and staff. In 
this way, clients have many 'chances Jto examine themselves, their 
abilities and interests, and .to clarify values and set goals that reflect 
their own unlqup personalities. * - ^ 

The material be^pw presents detail for each planning step. 

4 

Step 1: Individual Assessment and Goal/Competency Setting 

Assessing what jflpu have to start with is an important first step. Using 
a variety of information^ sources— including participant self-assessment, 
staff observations, performance reviews, and 'past .transcript data—the 
client works with staff to address questions like these: 



What have I already tried and'know that I can do? 
What \do I like to do? 
What am I capable of doing? 

What is* important to me now and for "the future? 



A: 



* • / * 9 

Whatever the initial assessment 1 * purpose— whether/academic, career or* 
personal, growth — its major focus is where the* part&cipant has come f rom, " 
where he or she is ri9 ; w< and what kinds of things look appealing in the 
* future. Special attenfcibiTTs given to what participants know- about 
^themselves and their unique characteristics. Competencies- in each of the 
four, domains may be set as goals— or the participant may work on only one 
competency area. ; 



Step 2: Selecting Activities *to Attain Competencies 



Fbllowing the initial assessment and goal-setting* phase, participant and 
staff negotiate goals, activities and performance criteria. They select 
resources 'that fit the individual's identified needs and expressed 
interests. An important staff responsibility at this point is to help 
the client creatively design a plan that will meet career and personal 
development goals. 

The participant wo*ks with staff to make the "first several goals 
short-term and highly specific. At the same time, they discuss how 
achieving these goals relates to long-range career and lifetime 
aspirations. These latter goals are usually very broad and tentative; 
most programs hesitate- to push clients to decide on a concrete and 
guaranteed "lifetime* career. Rather, staff, introduce clients to the 
need for continually making and refining long-term goals as an inqx>rtant 
aid in making decisions about the present. As clients experience 
Successes and failures in achieving short-term goals, their interaction 
with learning environments and resource people helps them refine broader 
future plans. At the same time, they develop the ability to consider * 
more options and to respond to changing concepts of self and changes in 
the world. 

Throughout, the participant is encouraged to .speak up when particular 
activities prove to be either too difficult or nonproductive. The - 
client's ability to renegotiate the specifics of the career plan is vital 
to the success of negotiation as a guidance process. 



Step 3: Ongoing Assessment of Competency Attainment 

* » 
i 

Ongoing assessment of competencies looks at what participants have done 
to achieve the goals they set for themselves . It may be based both on 
others! ^observations of their performance and on their own 
self -evaluation. Evaluative feedbadk and debriefing conferences are . 
recommended for any individualized program. The participant and staff 
discuss the assessment, its implications for career planning *and otonges 
in behavior that may be recommended. In this step, assessmenfcJndty be 
used 'to help participants understand how far they have come in the - ' ~ 
attainment of a particular benchmark and how far they still need to go. 
Ongoing ,assment can also be usedto identify areas in which additional 
work is needed. Ongoing assessment fteljis to modify activities as quickly 
as possible if a competency is not met. There is no sense in continuing 
_with an activity' if it is* not helping the participant attain an 
identified competency. t , — 

As much.as possible, the people who evaluate participants would be those 
•who have worked with them while the activities were being performed. On 
an informal basis, staff, employers and others have many opportunities to 
talk with clients about performance and growth. An important, part of 
assessment is its emphasis on the person's increasing ability to reflect 
cm first-hand experiences— to see the results iff , decisions made and to 
assess the effects clfibices have on selff^concept? , and lifestyle/career 
aspirations. * 




Step 4: Revision/Adjustment of Goals > * 

After each program activity and during periodic assessment or evaluation 
interviews, the .participant examines benchmark attainment, reviews 
abilities and whether interests have changted.^ The participant may wish ? 
to modify career and life* goals. As the participant refines or- modifies 
decisions, the^ycle repeats 1 itself and the student again meets with a 
staff person to establish desired competencies. ' ' *k 

As participants in any employment program gain experience in using the 
cycle of irfitial assessment, negotiations ongoing assessment and 
integration, they are encouraged more ana more to take the initiative in 
arranging meetings and proposing terms for negotiation. They begin to 
structure, plan and' then negotiate ever larger segments qf their 
activities and to take greater responsibility in meetifig terms and target 
dates and accepting the consequences of their behavior. Part of 'their 
learning should be the fact that employers, for instance, may not be as 
willing to negotiate on some of these issues as we might like! 

As participants become more skillful at negotiation, however, they come 
to understand the" benefits of the process, internalize it and transfer it 
consciously, to other life situations. It becomes their system — one that 
works for them again and again, 'one th^t makes personal success more 
obtainable ota short-tenn and long-term bases alike. ' • r 



IV. SOME TECHNIQUES YOU MIGHT USE ^ 

/ ' 

the techniques below have beeji used by other practitioners to carry out 
an individualized planning process. 4 . ' , 



Using Assessment Information 

• # 
To serve as a baseline for negotiation with participants, diagnostic 
inf ormatibn for each incoming studfent can be gathered on a single Profile 
form which, summarizes transcript data, testi'ng..results and student/staff 
interviews. _ . 



Transcript Data 



[ Before program 'entry, a staff person may wish to gatfher transcript or , 
other prior data on incoming participants and translate it to a fotfm 
usable by staff. This information may be kept on a memo flipped in the 

^client's file or summarized on the Profile form. 




* v to 

Basic Skills Testing , t ) 



/ 



Upon program entry, staff may 7 choose to begin assessing educational* 
skills, since these are the competencies frequently* cited as important to 
employers. Staff should try to spot participants who might, need 
additional testing early so that impropriate Educational Skills 
activities can be negotiated as win as possible. 

» * * 

Carreer Development Assessment 

On program entry, participants typically use instruments and systems 
designed to help pinpoinj^fche kinds of \ occupations that might interest 
them. Using -these materials early lets them make ^tentative selections of 
jobs they would like ta explore or prepare for. This enables staff to 
make sure there are ^sufficient work;^sxperience or' training opportunities 
to accommodate interests. 



^Participant Self -Analysis * * > 



How clients view themselves strongly' affects their progress and general 
adjustment to the program. They b'a^be asked to describe themselves in 
terms of ^thelr interests, pref erre&^iearning environments and types of 
learning, they enjoy . They might aitfo describe problems -they perceive, 
in their study habits and rate their abilities to* caraiiete assignments 
-independently. This self-analysis should'take^ place in private ? 
interview^ wibv^a staff member who uses a cMecJc-off sheet .to help guide 
£h% interViev^ aftd to record responses. * 

Careei P l^inj.ng Co nferences ' \ : 



The backbone of a responsive program consists of numerous observations 
made daily by staff and employers. It is .through -sharing these 
observations in career planning conferences that participants ajid staff 
gain «lrealistic pictures of client abilities and needs. Most observations 
made during such conferences should be filed in the staff member's 

personal .notebook and shared with other staff ;on an informal Jbasis. 

*" . ♦* * . '• * 

• » ft 4 A 

Early -during program orientation, *1^Jie job placement or work ekp^rientte • 
liaison staff member should meet individually with participants to 
counsel th^m, on their selection of sites for career e^cplorations (if*this 
option is available) . ^Results of career interest tests/ How* accurately- 
they reflect what the client feels about possible jobs, and any. personal 
'observations the staff person* makes are filed for Use in' information 
sharing meetings with other staff. * . 

, " ' , I * t * *; 

The adyisor might observe the client's initiative in using career ** 
research materials and in selecting sites tot explore, noting which 
clients might need more help than" others to get over their initial. f 
hesitancy at making contact with employers, or accepting job 
responsibilities** ^ . •* 



Continuous Participant Self-Evaluation/ 

Participants typically go through a qoritinuoils kind of setf^ 
assessment and analysis as they work with staff to develop individualized 
plans* The mote participants are involved in) planning their own learning 
and seating t&eir own performance ^criteria, the more proficient at 
self -evaluation an^self-assessment they will become* Staff, employers, _ 
and other resource people can aid in this process by giving participants 
regular feedback and advice regarding what they see* 

V*' , - 

V* STRATEGIES FOR PERSONALIZED LEARNING 



The attainment of benchmarks aad competencies will be difficult, and 
probably useless t if the educational activities do not meet the needs of 
thdT individuals they are aimed at. The best learning activities are 
personalizes* There are a number of ways staff can personalize 
activities designed for participants in competency-based employability 
skills progratos. 

In general, such activities should contain the following elements: 
/ !• Specifies rationale, activities, resources, and products-criteria 



/ 
/ 



2* Cleanly defines target dates 

3. 16 performed at, or involves experiences from, job sites/in the 
community ^ ±, 

4« Involves participant in active use of the community as a 
resource pool (people, places , materials) ~ 

<& 

5* Involves participant in frequent "learning by doing" experiences 

6« Tailprs goals to individua^needs, interests and abilities, 
balked on assessment information 



; 7. Measurabl*e performance criteria negotiated individually. 
-* 8. ^ Emphasizes career planning and lifetime learning 
9. Invdlvtf* community people^in the evaluation process t 



w Personalised Instruct ia/rf X Definition * 

* * * * 

Any instruction or training Which is planned # to meet the individual needs 
.of a participant can be personalized by; 

- Proceeding at the person's own rate of progress and. achievement 



- Varying the ^amount and kinfc of instruction based on 

.participant's ability level, interest level and personal learning 



style as diagnosed by the staff member 



- Varying the leafhing setting in and out of the typical classroom, 
in , various grouped and independent 3 instructional modes, <$ased 
on'th$ diagnosed learning needs of the person^' 



* it 
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v individualized' instruction is 

. NOT NECESSARILY*' K BUT USUALLY INVOLVES: 



One person -working with 
one instructor 



o Some one-on-one time, especially for 
planning learning and evaluating 
achievement 



In small groups 



o ' Small grouping strategies and 
e activities, including peer tutoring, 
even in large group settings 



Self -paced 



o . Time flexibility, so that the 

student masters one competency in a 
particular topical area before 
proceeding to the next 



Self -directed 



o Participant/choice, with staff 
'^.jg^idance^of learning materials and 
activities , 

o* Client participation in the' planning 
and evaluation of learning 

o Seme independent work 



Out of the classroom 



o f Scme out-of-the-classroOTi 

'activities, linking ".theory" with 
£he rest of the client's life 



Ungraded * ■ 



\ 



0 Some flexibility in grading 

# * procedures to give participants 
credit for previously mastered, 
1 skills and to focus attention on • 
(fjrlterion skills or competencies 

I o Provision of' f eedback to partici- 

pantJ2_pn their- performance in a m^e 

1 extensive way A than typical grades 
providfe - \ :\ \ 4 



An individual learning plata 
(sy'ch as'tlfe Career Plan) 



V 



o Flexibility in /assignment a and- 
o material^ and strategics to meet the 
( needslof participaxits^individually 

u 





FIGURE 5-3 x INDIVIDUALIZED INSTRUCTION 



0 



There are several methods to choose from in providing personalized 
services. * Three clusters are described below. It is understood that 
there will be fiany variations within each* Another assun^tion we make 'is 
that counseling and guidance resources are available .continually as 
clients discover- tieeds to change their plan because of shifting 
circumstances • 

The three clusters are : , 

Group or class experience?^ - where 4 as many as 20 to 30 persons 
gather regularly for learxiing or training activities planned a'nd , 
directed by one or more staff persons. - ' 

( 

2> Pre-planned projects - where either individuals or small groups 
trork together on a common set of objectives and ' 
activities witfi^a staff member or knowledgeable resource person 
nearby. 

3 j Individual project - wherfe the client works out a personally- 
tailored learning plan or contract to accon^lish unique objectives 
based on previously identified needs, interests and abilities. A 
staff .member or resource person may only need to be consulted 
occasionally. Typically/ several resource persons will nyd to 
be involved. • W 

* - . ^ 
Listed below are questions to ask in deciding which strategy to use, when 
to/use it, and with what clients. 

St*** ' * • -> 

1. , Are staff members equally skilled 'in working with large groups/ 

small groups and one^-on-one? *• . 

2. Are there eriough staff to offer more than one option* 

Space {Facilities) 

.1. Is there flexible spacfe to accommodate different learning modes? 
2. Are physical surroundings conducive to individual vs. group 
r 'activities (e.g. lighting; temperature noise levels)? 

{" f : 

Time \ » 

1. Is the program bound to, produce minimum outcomes in a' maximum 

•time frame? / 9 t ' . # * ' 

. 2*. Are hours in a day or week 1 flexible? , 4 

Materials and Equipment . / 

1. .Are materials and equipment readily accessible and adeqoitely 
\ Supplied? ■ h. 

2. Are they up-to-date and cocqpatible with todays rapidly changing 
tecjhnology? x , 7" • ^ 

3. ,Is there someone monitoring their availability and managing the 
4 flow? * , v 



79 • 84 



Transportation * • . \ , ^ 

1. Can participants safely reach their education^ and training 
sites 9 and with; relative ease?^ 

2. Are costs a major 'concern in this instance? 

v " - Community Resource Persons 

!• is use of knowledgeable experts possible either in the classroom 

^ situation or by participants at worksites? 
2. " Have those volunteer 'resource persons been trained? » 

• 7 ■ 

Community Resource Sites ( 

- Hasvthe use of ccntmunity sites (any pl^ce where work is being 
done) been considered as ^a way to achieve other competencies 
than those related to job skill development and placement? 
2. Have volunteer tutors^*, mentors, , or supervisors at work sites 
* been prepared for their roles as instructors and to certify that 

learning and skill development hav^taken place? 

Programs that ,h£ve adapted individualized education and* training 
strategies typically find them to be far more effective than 
group-oriented/ timfe-bound systems. While costs are comparable, and may., 
even be greater in the planning strages, the overall outcome will more 
them compensate. * 



Chapter Six: RECEDING AND REPORTING 

* f • 

!• OVERVIEW 



The basic management topi, of a competency-based approach is a - ^ 
recordkeeping system which allows staff to determine individual progress 
and to monitor overall "effectiveness of programming • Because of the 
data-dependent nature of a competency-based system/ it's not suprising 
that the most frequent staff complaint will be that recordkeeping tasks 
take too much time away from other activi ties The volume and frequency 
of these complaints wi 1 1. war rant ^harS IooJq at efficient ways to meet 
recordkeeping demands • 9% 

In determining the recordkeeping needs of a competency-based^ system/ 
three questions must be addressed. * Who- is likely to need infotmat'ion 
about competency mastery? From where will the data come? What use will 
be made of the infopaation? ' , ' 

A major point to remember in designing recordkeeping systems .for 
competency-based programs is that records must bfe kept by competency for 
each individual. Keeping track of time in a service or placements in a 
job is not sufficient. 



- II. AUDIENCES 

o - * ' * ♦ * 

There are five maia audiences for" infolnnation compiled througi^ • 
record-keeping processes: (1) staff, (2)° participants, (3) employers, 
(4) funding agencies and (5) the community-at-larga. Th/$ basic 
information for all audiences is the same^vhat individuals are achieving 
what conqpetencies—but the detail and forpai for presenting the ' 
information varies by audience. 

o" Staff need information** for two ^t^poses:; /D to plan activities 
for individual participants" and T2 ) to improve, the overall - 
quality of the program. To plan activities for participants, 
staff members must know which competencies have been achieved 
. and which are yet to* be achieved for each individual. To 

y ^irorove the overall program, the staff must know which 

competencies are being achieved by- all or nearly all of the * 
participants, and which are being Achieved by relatively few^ 
. individuals • This knowledge' allows staff to concent rate^ 
improvement ef f qrts on those competencies, that are not being 
achieved. > 

a. * 

ip Participants need to kfiow how they are progressing. They must> 
know which cons>etencies they .have passed and which one* they , 
wist develop. Individuals must take responsibility for their 
. own development even though it is i\pt easy to do so. If 
individuals are not responsible and actively involved in 
managing their development/ progress will be much slowest. 



T 



O' 



Employer g must know what cons>etencies each individual 

tfoswsses. ' This information is necessary in order to make 

♦hiring , r decisions. Good information will help both employers apd 

employees (-participants) . Employers will, hire people who have 

the specif ic jsoofcetencies needed for their job. Individuals 

will not be placed in jobs for which they are poorly prepared; 

they will Jiave a greater charfBe of success in the' job they do v 

get. * • • 

* 

Funding agertc^es^ need information to determine whether an agency 
is meeting management goals agreed upon r f or the program. They 
need to know how many and which individuals' are achieving 
various com£>etenciea. 

The- cjennunity-at-large needs td know how the program is 
performing. Is the program doing what it said it would do? 



III. SOURCES 



Data will come from a variety of source's, depending on the assessment 
techniques „ used. Major sources are test scores, enployer ratings and 
'records and staff judgments. Keep in mind that the data should be 
recorded by competency for each individual* 

m A recordkeeping system can be developed only, after the competencies and 
standards are explicit and the sources of data are known. The first step 
in setting up a recordkeeping system, then, is to list the con^etencies, 
standards and data sources. Samples follow: y 



Competency Area p Cocg>etency Indicator 



Standard 



Data* Source 



Pre-Employmeni 



.Educational 
'skills 



Summarize jdb 
market condi- 
tions 



Communicate 
through writ- 
ing and 
speaking 



Name sources 
of informa- 
tion About 
job market 

Write legi- 
legibly 



At least 
three f 



Others can 
read and . 
interpret 



I teat 4 from 
unitfeest 

r 



Teacher judg- 
ment of 
Writing on 
forms and 
practice 
writing 
assignments 



Work Maturity 



Demonstrate 
Appropriate 
work per- 
formance 



.Do enough 
work 



■Quantity 
acceptable 
,to .employer 



Empldyer ' % 
judgment as ' 
— reported on^— 
work maturity 
checklist 



On , 
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There may* be several indicators for each competency and performance on 
all indicators must be documented before comj>etency has been achieved. A 
recording "form may look like the one shown below* When oon^etencies and - 
indicators are known, form* can be pre-printed to save time.. Such a 
record must- be updated frequently, perhaps even daily. , When'columns ar$' 
totaled, this form shows overall program performance. It also -shows 
progress by any individual.' < * 

i • . : 



% " 4 > 1 — 

work maturity . ' . Program ' 

COMPETENCY -DEVELOPMENT ' . . • Date 


< 

0 

1 

Participant 


DEPENDABILITY 


V 

0) 

o 
c 

4 0> 

< 


Punctuality 


Notification ^ 


* * 

'Mo 
W 


Grooming . 


Dress 


8 

W 

s 


I Quantity 


Qualify 


Conseqbences of Errors 


3 

H 

w 

i 

o 
z 

H 


Supervision j 


Co-wbrkers 


Cooperation 


w 

Eil 
Q 
D 
^ 
H 


Interest 
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It is probably obvious that several different data* sources will produce • 
information usable in tracking* progress of < individuals* ^he dat;a fro© 
the various/dfcurces must be transferred onto a single fontf. 
Computerizing the recordkeeping system wifil help reduce the clerical load. 

Uses and Reports • % J . 

Tfce uses to be made of the information must be matched with the report 
format. Individual profiles are useful to staff and participants. A 
sample is shown below. » ♦ u 

c * * 

* 

EMPLOYABILITY COMPETENCY * 
DEVELOPMENT PROFILE 




Participant * Date . 



•Service Provider ^ 

' . 



Competencies Complet ed In-Progre ss 

■ * 

- PRE-EMPLOYMENT 

• * r * 

/ Describe Personal Characteristics 

How to get information . * / 

• • ' ' * / 

List interests " * / 

List abilities 

List short- and long-range 
* g^tls 



^his type of report helps to focus attention on the skills Ainder 
development. Comparing reports ffcom various periods will show how well 
the individual is progressing. 



/ 



£taff members ' need summary information in order to check program 
performance. A summary report may look like the one below*. 



a ■ ■ . ■ . 
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EMP^OYABILITY ^DEVELOPMENT 
SUMMARY RfPORT 



Service -Provider 



This report cpv&cs therjxqriod ftrpm* r ^ 



Completed 



In-progress 



Competencies* 



Number Percent Number Percent 



PRE-EMPtGJfMEOT „ 

Describe Personal Characteristics , 
How to get "information 
' Lilt ipt§rests ► 

List abilities'* *. 

List short- 'and long-range goals 



10 


<20 


40 


80 


48 


96 


2 


4 


30 


60 


20 


40 


23 


46 


14 


28 


10 


20 


10 


20 



This type of report allows staff to see which indicators are not bein^ 
achieved and to focus ofi determining why progress is not being made* If 
necessary, .the program, can bfe adjusted* 

Employers are- most interested in the competencies that individuals h&ve 
achieved* They are probably mfcst interested, in work maturity, 
-educational skills. and occupational skills* The reports to employers 
shduld be in simple, straightforward language and may take the fprm of 
certificates* 

Funding agencies need data on program performance that is much like what 
the staff needs 'to J^ust the program. The difference is that**the 
funding agency ma^Hnt information on competencies only, not on" 
indicators* Als4^Khe information should be displayed with the 
management goals agreed upon by the funding agency and the service 
provider* Samples ,of such reports are in Chapter Five, Managing A 
Competency-Based Program. m + 
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The ccnnaunity-at-large needs general information about program q 
performance. Reports should be by competency and relate to management 
goals. A sample report is in Chapter Five. 

IV. PRACTICAL .RECORDKEEPING 



Typically/ it will be staff members who* are asked to accomplish 
instructional/ assessment and record tasks. Since most agencies have 
existing information 'systems in operation, it is important that the 
competency recordkeeping system and existing procedures mesh and do nqt 
duplicate collection and recording procedures. 

Computer systems have been touted as a means of facilitating good % 
recordkeeping for competency-based systems and they* can serve a^ valuable / 
function, particularly in the derivation of group data frola the g I 

information about individual students. However, a computer does not* \ 
reduce the work of data collection: designing, administering, scoring and 
recording the assessments of individual participants; so a computer will 
not 'appreciably reduce staff workload. . \ 

Recordkeeping by exception , could aisp reduce the teacherXp recordkeeping 
burden. That is, instead of recording successful mastery of the 
coripetencies, the data collector would orfly record the' 
deficiencies— -con^etencies that were not mastered. Most teachers ha^e 
responded tb.this suggestion by saying that they appreciate the positive 
reinforcement aspect of marking mastered competencies for each student. 
So, while recordkeeping by exception would reduce the number of entries 
that the staff must make for each participant, instructors are reluctant 
to discard tjie individual reward factor implicit irAcompentency-based 
systems. \ 

A final suggestion is that staff — those with responsibility for 
implementation— -be heavily involved in 'designing the recordkeeping and 
reporting system. This will maximize chances of success. 



v 
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AFTERWORD: READY TO WORK IN A COMPLEX JOB WORLD 



This handbook was. created in response \to a growing interest in 
competency-based education and training. programs. .Employers frequently 
•ay they viahTnew employees would come to work with the Vight" attitude 
and skills so that productivity id not lost. The techniques described in 
this g manual are based on real' examples of prbgrams* that are underway in 
communities large and small across the nation* The key ingredient to make 
these strategies work from your point of view is a willingrtess to try. 
The roadpap is provided here, but the routes you choose to take* are local 
decisions* To get the best advice on directions to explore, ask around 
for people who have made similar journeys^ Their recommendations and 
program. models will provide a, reality check for the guidelines collected 
here.*' The approaches may vjky but the outcomes should be the satae: to 
help youth and adults alifce be r$ady for today and tomorrow's 
ever-changing work of wqfk. • 
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A. WRITTEN RESOURCES^ 



'l.' /APL 'Series; Coping in Today's Society (The) > New York: Harcourt, 
Brace, Joyanovich, Inc., 1980. • \ % 

The Adul tpe rf oraanc e £e ve 1 (^L) series is a performance-based * 
curricultim for aduitfc that focuses oir skills for getting along in . 
today's society* Tlie series ^consiptir^of the following, volume^: 
% Government and Law, Health I. 'and II; Gonsum^r Economics X,, ll and' 
n III; .Occupational Knowledge and Community* Resources. The' volume on 
Occupational Knowledge contains ♦material closely related tp the », * 
competency: areas ^of Work Maturity and Pre-emplpyment. ' . 

2 a . "A Methodology for Reading 4 Skill Improvement in Vocational Secondary 
" Programs,? Salem, Oregon, School District; Vocational* Technical 

Educatioii Division, School , of* -Educatiorf; Oregon State University; 
v s and Oregon Department pf E&fcat ion,., June 1981. > v 

,3. "A* Summary Report of th e" V lceHfiresident 's- Task Forqe o>Jfout^.-j: 

anployment,"' Washifigfon, D.C.Y'The White House, - 1980v 



-4. Chatham; Karen M., Jphnson, Jarates N. and Peterson, Robert JM* *The' 
Concept of toployabilityj^A Review of, the Issues , " San Francisdo, 
©California: Far West Laboratory for Educational Research and 
Development, July 1981 (draft). * 



5. "Concept Paper on the Consolidated Youth Employment 



?ram 



Demonstration, " Washington; D.€. : U. S. Department ^of .I^bo^ 1980. 



6» "Consolidated Youth Employment Program: Promising Confections for 
Youth : Field Observations and Policy Analyses of Key^ PrpgtfSlm . 
• Issvies," Washington, D.C.: Osoro and. Associates, i?8n n: W 



7. "CY^P Special seminar: Competency Assessment/Benchmarl$|f " 
Development," CYEP Technical Assistance Letter #24) 0. S* Department 
of Labor, anployment and Twining Administration, 1981. 

8. Frye, Charles H., "Tracking and Reporting School-Leaving . 
Competencies," Portland, Oregon: Northwest .Regional Educational 



Laboratory 



9. Hamilton, Steven, "Youth Work Experience and Employability : 

* Literature Review and Conceptual Framework," The National Institute 
of Education, July '1980. 
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16. Johnson, James N, "The Concept of Work Maturity, "San Francisco,, 
California^ Far West Laboratory for Educational Research and ( : \ 

' Development ,'-'July 1981 (draft). 

11. McCaslin, N. L., Gross, Charles J. and Walker, Jerry P., "Career 
• Education Measures: A Compendium of Evaluation Instruments,* 1 
Research and Development Series No. 166. National Center for 
Research and Vocational Education.. The Ohio State University (1960 
Kenny Road, Columbus, Ohio 43210), 1979. 

* i 1 1 

-12i McCaslin, N. L. and Walker, Jerry P., "A Guide for Improving Locally 
Developed Career Education Measures, "^ Research and Development 
<SeriesNo. 167, National Center for Research in Vocational 
Education. The Ohio State University (1960 Kenny Road, Columbus, 
• ' Ohio 43210), U979. 

\ 

13. 'Malak, Sharon, Sjpirer, Janet E. and Land, Brenda Pellegrini,, 

"Assessing* Experiential Learning in Career Education," Research and 
Development Series No. 165. National Center for Researph in 
Vocational Education. The <&io *Stdte, University, 1979. 

14. Michel,. Ann C. , Reicher, Rita L. ; Cama, Vincent, McMaster, Donald 
J., Benchmarking: A State-bf-i:he Art Review / 3 vol., Syracuse, New - 
York: t Syracuse "Research Corporation/ May 31, 1981. 

, ' . t . 

15. Nedwek, BrianP., et al. "The Vocational Exploration Demonstration 
'Prbject: An Analysis of the 1979-80 'Academic Year Components, " 

Project Report Submitted to^the U.S. Department of Labor, St. Louis, 
Missouri: St. Ip*a£ University, 1981. » 

<16. Nickse, Ruth, et. > with McClure, Larry, Associate Editor, ^ w 

Competency-Basejn Education: Beyond Minimum Competency Testing , New 
York: TeaSh^ftr College , Columbia University, 1981. 

17. ^elz, Nina, "The Teaching 'of Etaployability Skills: Who's 

Re&ponsible?" Columbus, Ohio: National Center for Research in , 
Vocational Education, 1980. 
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' ^ B. SOURCES' OP INFORMATION AND MATERIALS 

Adamsky, Didi, "Vocational Education Department, Templ^tJJniverlity, * 
Philadelphia, Pennsylvania 19122. 

**Modtilesrto a§9*st irf developing competencyrbased Instruction in 

vocatidnal education. These relate particularly td the y 
' Occupational Skills competency area. * \ 

■ I - * 

••Fair Break," Control Data Corporation, *&x 0, Minneapolis, 
Minnesota 55440. 

♦ 

♦ ♦••Fair Break" is a program, utilizing the computer-based '£ LATO 
system to train the unemployed for success in the workplace. The 
PLATO system is competency-based and includes self -pacing material 
'designed to teach basic skills. e 

M»thods'«and* Materials for Teaching Occupational Survival Skills , 
Illinois State Board of Education, Curriculum~$ublications 
Clearinghouse, 76B fcorrabin Hall, Western Illinois University, 
Macomb, Illinois 61455. 

♦*This i^'a 650-page resource guide for teachers relating primax 
to the "human" side of work. It consists of 14 module^ .and* relat 
to Pre-employmenti and Work Maturity competencies. . 

y - * T , 

Minnesota Curriculum Services Renter, 3554 Whitfe Bear ?Ave*nue, White 

Bear Lake* Minnesota 55110. ^ 

* 

>*Competency-based instructional materials fcrom 25 -vocational program 
areas. These Relate particularly vto the Occupational Skills 
Compentency afea. * - „ • • • * 

r t ' 

O^en Entries /'The'fcent.er or Studies in Vocational Educatiorff „ 
Florida State University; Stone Building, Tallahassee, Florida 32306.* 

♦♦This newsletter* provides resources that relate, to 'all. £our 
competency areas. ' > . 

Occupational 'Training Guide for the Jo*b Corps , ET%Ha.ndbo6k No* 325, 
U.S. Department of Labor, Employment and Training ^Administration, 
1976. - '+ *' ■ - • »• % 

*• • • / * 1 /. / * f . 

his ^ i : de relates particfularly to the Occupation Skilly competency. 




7. Schoolfutures, Inc. , Educational Support Center, 2092 East Main 
Street/ Hillsboro, Oregon 97213 

i 

**Sfchoolfutures, Inc. is brokering company that matches client nfe^ds 
with a variety of commercially available materials in vocational 
and job-related education. Materials available through l^**** x9 
Schoolfutures, Inc., relate particularly to Pre-employment? and Work 
Maturity competencies. * _ 

8. Lee Schuff, Educational Resource Cejvte^fFox Valley Technical ^ 
Institute, P.O. Box 2277, Apple tori^ Wisconsin 54913. • ' 

♦♦Audiotapes and videocassettes supposing copipentency-based 
vpcational Education. These relate particularly to the 
Occupational Skills competency area. 

9* Vocational Training Guide , U.S. Department of Labor, Employment and 
TrajL^Ang Administration, Office of Job Corps. 

¥ 

**This guide cap .be of assistance with the Occupational Skills 
competency area. 

10. Vocational-Technical Education Consortium: of states (V-TECS), 
Commission on Occupational Education Institutions, Southern 
^Association of tolleges^and Schools, 795 Peachtree Street, N.E., 
Atlanta, GeorgiaT 30308. 

^*#V-TECS produces and sells materials for use in vocational-technical 
education curriculum development, program planning and evaluation. 
Materials consist .of worker-validated catalogs of performance 
objectives and performance guides. These materials relate tb the 
Occupational Skills competency area. 
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List of Prime Sponsors Participating in the Consolidated ^Youth 
Employment Program (CYEP) 



\ ' 9 

PENOBSCOT CONSORTIUM . ^ 

}33 Illinois Avenue 
P.O. Box 1136 

Bangor, Maine 04401 X 
(207) 945-9431 

MORRIS COUNTY EMPLOYMENT & TRAINING ADMINSTRATION ' • 
3 Schuyler Place * , . 

Morristown, New Jersey 07960 * ' */ 

(201) 285-2762 

ERIE COUNTY DEPARTMENT OF EMPLOYMENT A to TRAINING 
G. Daniel Baldwin Building 
1005 State Strc^ 

Suite 1215 . • 

Erie, Pennsylvania 165*01 
(814) 453-5838 

ESCAMBIA COUNTY &TA 
Town and Country Plaza Building 
Suite 414 

Pensacola, -Florida 225VT*^ 

(904^ 436-93j67 / s \ 




SOUTH FLORIDA EMPLOYMENT AND TRAINING CONSORTIUM (SFETC) 
7880 Biscayne Boulevard 
Miami, Florida 
(305) 638-6819 

ROCK ISLAND COUNTY OFFICE OF CE^A .PROGRAMS 
416 19th Street 
Rock Island, Illinois 61201 
(309) 793-5227 

CITY OF DULUTH-CAREER DEVELOPMENT CENTER 
206 Vfest Fourth Street 
Duluth, Minnesota 55806 
(218) 723-3496 

CENTRAL TEXAS MANPOWER CONSORTIUM 
c/o Central Texas College 
Highway 190 West 
P.O. Box 108 
Killeen, Texas 76541 
(817) 526-1320 
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PENINSULA OFFICE OF MANPOWER PROGRAMS 
2017 Cunningham Drive ' 
P.O. Box'7489 ' 
Hampton, Virginia 23666 
(804) 838-5206 



BALANCE OF STATE - KANSAS 
Department of Human Resource s/CETA 
708 Jackson 



P*0< fr Box 1556 
•Tojjeka,' Kansas 66601 
(913 ) 296-3031- 



V 



CITY OF TORRANCE - OFFICE OF CETA PROGRAMS 
3231 Torrance Boulevard 
Torrance, California 90501 
(213) 32^-5310 X?31 

YAKIMA COUNTY CETA 
'2011 South 64th Avenie.^ 
Yakima, Washington 98903 
(S09i 575-4252 

BALANCE OF STJATE WASHINGTON 
anployment and Training Division 
1007 South Washington Street 
♦Olympia, Washington 98504 
(206) 754-1055 \ % ^ 
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SAMPLE COMPETENCY DEVELOPMENT PLAN 



In order to clarify the purposes of th^ youth employability program of 
the balance -of state CETA prime sponsor/ to improve communication and 
cooperation among^ag^ncies and'organizatibns interested in youth 
employabilty development/ and to improve accountability of youth > 
employability programs/ a competency development project is being 
undertaken. The specific objectives of the project follow: 

- To establish a complete list of employability competencies 

To establish relative importance among the various competencies 

- To determine responsibility for providing services and 
activities that lead youth to competency development 



Data Collection Analysis and Reporting Procedures » 

Because of the nature of the community and a desire for widespread 
community involvement in the competency development effort/ the following 
procedures will be used to collect information: 

Phase I • * 

- A series of five open' forum meeings will be held in key 
'locations throughout the area. 

\ 

« \ 

A group of no more than 25 persons will attend each meeting. 

Each person will be invited by letter and asked to confirm 

attendance. Alternates will be invited for those who cannot 

attend so that thefefare 25 persons at eadh meeting. 




The group at each meeting will be representative lof all 
interested agencies and organizations as listed below. ^ 

Each meeting will be four hours in length so that participants 
can drive J.n, meet and drive back in one day. 

A trained facilitator will be identified and asked to facilitate 
all 1 five meetings* An honorarium will be paid if necessary. 

The initial part of each meeting will be devoted to an 
explanation of the project and group building activities* The 
remaining portion of the meeting will follow open forum 
processes with small groups working together and then reporting 
back to the large group. 

i 

The ideas from the small 'groups will be synthesized together 
into a single listing of employability competencies . { The entire 
group will be involved in the synthesis process. 

The ideas from the five meetings- will be synthesized into a 
single }ist of e^loyabiilty competencies. The list will be 
distributed with a letter of thanks to all participants and , 
youth advisory council members. The list will provide the basis 
for the .survey in Phase II • 
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Phase II 



% Using the competency .list generated through the open forum* 
process, an opinionnaire. will be constructed to determine the 
relative importance of each competency and to discover various 
agencies and organizations that provide services or activities 
that lead to the development of eaqh con^etency. There will be 
two basic questions asked in relationship tfe each competency. 
They are: 

1. How important is the competency to getting and holding 
-p employment? 

,2. Do you or does your agency or organization provide 
services or activities tpiat help youth develop the 
c cmpetency? I * 

The survey form will, be developed by a consultant, reviewed by 
the youth advisory council and pilot tested with ten individuals 
before the final form is printed and distributed. 

The survey will be distributed to all or a sample of individuals 
in each group listed below under "Participants. " There will be 
a postage-paid return envelope included for the convenience of 
the persons completing the form. A cover letter will explain 
the competency development effort and urge the persop to 
complete the opinionnaire. The cover ^letter will be signed by 
the governor or lieutenant governor and will be addressed' 
individually £o the person being masked to complete the form. 

Question One on the survey will allow response on a five point 
scale of importance in which* five (5) will represent very 
portant and one (1) will represent: not important. Question 
o wilj. be answered either yes or no. The response will be to 
check the appropriate spaces. 

r 

The relative importance will be determined by the average rating 
by the groups. 

Results Of the si{rvey will be analyzed for $ach group and tor * 
the combination" of all groups. The report will describe areas 
of agreement among 4< ^Dups as well as areas of disagreement. 'The 
competencies will be presented in rank order from highest to 
lowest average score. ^ 

The answer to Question Two will be used to discover agencies and 
organizations that are providing services and/or activities for 
SMcific competencies. The analysis will show which 
competencies are being worked on by one or more agencies or 
organizations and which ones are not being developed. The 
report will show which agencies or Organizations are working on 
the same competencies, thus setting up the possibility of 
cooperation. 
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The results of the survey will be sent to each person who 
completes the form and will be -publicized widely through news 
releases, radio*&nd TV announcements and direct mailings vtp . 
schools, Chamber of Commerce members and service club members. 



Participants 1 ' • 

individuals representing the following key .groups will be Included in, 
both phases^.of the projeptj 

Youth ages 14 to 18 * Lists will be obtained from schopls, 
employment- service, alternati"^e nonpublic schools and CETA prim 
.sponsors. • 



Parents of 



as for youth 




List will be obtaine<Jjfrom the same sources 

•.-I - ■ . 

Employers , Lists of contact persons will be obtained from 
Chambers of Commerce^ .vocational education programs, service 
clubs, government agencies and .businesses. 

School staff . List will be obtained from schools." 

"Staff from CE,TA, employment -service, and alternative nonpublic 
schools . Lists will be obtained from the agencies. 

Representatives from regional' economic developmeifc councils . 
Names of contact persons will i>e obtained frojn the councils. 



Anticipated Final Product 



The final product^will be a. list bf employability competencies \ Three 
levels of specificity in expiessron will be included. The three levels 
are competency areas, competencies and competency indicators. Standards 
(benchmarks) will' be set in'a later effort. Anexample of the 
three-level system follows: - 4 



Competency Area : 
Competency : * 

Competency Indifcators ; 



Work Maturity r , 
Demonstrate appropriate work 
attitudfe 
^ Shows interest, in the job 

- Observes regulations, policies 
♦ and procedures* 

- Cares for the property of others 

- Uses correct communications 
channels 

- Does appropriate share of work" 

- Shows a willingness to learn 
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/ * 
The final product will also have an explanation of the project and the 

use to be made of the list. The relative importance of each con^etency 
will be shown also. / 



Uses of the Final Product 



The-setraf competencies will ie used in the following ways: 



To cdmraunicate to employers the intept-of yputh employability 
programs / ' y . , 

To establish cooperative efforts jwith other agencies -or organizations 
in delivering services /or activities to youth 

- " To develop procedures, to assess competency development by* youth 

- To improve accountability for development of youth employability 

- * To improve procedures for documehting and reporting conpetency- 

development of individual youth 

Resources 

. The following resources have been allocated to the effort: 

$2Q00 from the balance of state CETA prime sponsor budget 
25 percent of the youth coordinator's time t 

10 percent of time from counselors at three secondary schools ^ 

Printing of survey forms at cost of materials by a local business 

T, Volunteer timej from Chamber of Commerce representatives from two 
chapters > 1 1 < „ 

* 

Timeline 



Start , 
Conduct open forum meetings 

Cgmpile results and send report 
Develop survey instrument 
Send survey 

Send followup to nonrespondents 
Analyze data w 

Prepare emc^ distribute final report 



October 15 
October 22 and 29 
November 3, 10 , and 17 
December 15 
February 1 
February 15 
. March 7 

flay 15 ^ 



(end pf sample plan) 
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SAMPLE COMPETENCY SETS 
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Work Maturity Competencies 



\ 



Competencies in this doolain represent the skills^and attitudes * essential 
to function as a valued employe • The skills and attitudes developed in" 
work maturity will enable participants to meet enqployer dfemands for* 
getting work done, getting alpng with supervisory and other personnel and 
taking the initiative for the benefit of the employer anfl oneself. 

the participant will:' 

1. 1 Demonstrate dependability 



o Be in attendance regularly 
o Be punctual 



2. 



3. 



► 4. 



5, 



o Notify others when absence or lateness is necessary 

i 

Demonstrate appropriate appearance 

i 

o Be groomed properly 
o Be dressed property 

I . • * 

Demonstrate appropriate work performance 

o Meet standards for quantity of work , 

o Meet standards for quality op work ' • v 

o Describe the consequences, of" errors 

Demonstrate appropriate interpersonal skills 

--6- -Relate -appropriately to dne or. more supervisors, 
o Relate appropriately to co-workers ♦ \ 

o Show willingness and ability to work cooperatively with others 

* 

Demonstrate appropriate work attitudes * " 

o Describe appropriate work attitude^ t 

o Show interest in the job ' N " 

o Observe regulations, policies and procedures* ^ • • 

o Care for the property of others and the employer 

o Use correct communication chapnels ** 

„ o Do an appropriate- share of work 

o Show a willingness to learn 



I - • 

6/ Demonstrate work, adjustment skills 

o Keep trying in spite of problems and setbacks 
o Resolve conflicts in constructive ways % 
o Accept things that cannot be changed 

o Work toward changes that are possible and beneficial to -both 
employer and employee / * 

o Adjust to new assignemen^s r 

o Work under pressure of deadlines and heavy demand periods ' 

o Use good judgment in emergencies crises and unexpected events 

7. , Demonstrate responsibility 

f € 

o Practice good safety habits 
o' Accept responsibility for and consequences of own behavior 
o. Follow .through on commitments < 
. o Check quality of own^ work • a 

o Meet deadlines 

8* .Demonstrate time management skills . 

* o Organize 6$id schedule tasks 
. o Set priorities for woz;k to*be done * 
o ^Plaft for unexpected events 

\ ' . ■ • • : . • 

9.' Demonstrate initiative and resourcefulness 

o Know what has to be done 

o svo work on own »with minimal supervision 

o Contribute to the organization by recommending better ways of 
• doing things » „ 

, o , Inform supervisor of unused' skills and abilities 



Pre-Etoployment Competencies 

* \ # ^ * t y . /- \ 

Competencies in this domain cover the knowledge and skills each CETA ' 
participant needs^ to make a variety of odcup&tional decisions and- to 
effectively seek and enter employment. -These competencies orient t'he* 
CETA participant %o the^work environment. The emphasis is on job 

.pljmning and the development of occupational self-control. '* • 

• * 
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The participant. 

1. Describe personal characteristics that relate to employment 

• " . r 
o Describe methods for gaining information ^about self 

A- 
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o Identify interests-, abilities, limitations, short- and 
long-range goals 

2. Explain conditions associated with work 

o Describe general characteristics of jobs and work 
environments (routine nrature of most wdrk, need for 
on-the-job teamwork, need for productivity and quality 
control, relationships between work and leisure, etc.) 

o Relate education and -experience to job entry and advahcement 

3. Explore job preferences , 



o Name preferred jobs J"* 1 

v 

o Locate information about preferred jobs 

o Describe preferr&I jobsj' (setting, hours, pay rates, entry 
level qualifications) \ 

o Match personal character! s^ics/ihterests/needs* with preferred 

* * A ohs J * % * 

*o Name jobs that could be entered directly and tnote preferred 
jobs that require more training of experience v 

^ Summarize ' job market conditions • 

» , o Name sources of information about the job market 

o Describe market character! sties iof preferred jobs - 
o Predict availability of preferred jobs 

o Select one or more preferred jobs for job search effort ,* 
5. .Locate job openings N * 

o Name hoping agencies and describe services available 

o List sources of information that can be used to find vacancies 

o Use information resources to find out about* job openings 

^ 6* Apply for a job .> - 

o Prepare a personal fact sheet (simple resume) 
o Complete a job application • 
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7. Interview for a job 

o Prepare for an interview. (rehearse, dress appropriately, etc. ) 

o Interact effectively with an Interviewer (maintain eye 
contact, -answer questions directly, avoid slan£, ask 
appropriate questions about the job, etc.) 

8. 1 Start a ^©Ineffectively 

o Adfce^t a offer appropriately (positive approach, direct 
^contact) 

, ©"Gather necessary * start-up information (when? where? resources 
needed?) 

o Describe fringe bexfc 




o Conplete paperwork related^o^Job^tart-up artd* fringe benefits 

o Calculate take-hane pay and budget it ♦ > 

* , o Obtain transportation to ahd f-ronl job v 

o Obtain needed job support services (day care, e£c.) 

9. Develop a job plan 

* * * * 

o Set short -and long range job plans 

_o Describe opportunities for. advancement *above entry level 
* s * 

o Describe techniques for getting ahead on-the-jofc (consistent 
high performance ,~ extra- effort, positive attitude) 

o Describe how to get additional skill training 

r * * - " 

o Relate income to immediate and long fcange personal* goals 

<? * 

10. Solve on-the-job problems 

o Describe union benefits and responsibilities of membership 

o Cite labor contract terms and grievance procedures 

o State the practical applications of labor, "civil rights and 
safety laws 

11. s Plan a jpb chang^# • 

o Cite reasons for changing job's " 

o Describe how to end^nqDloyment appropriately (letter of 

resignation, ' obta^nin£ letter of. recommendation from ^ 
employer, etc.) 

-o Locate job change advice 

o Plan a job change that will result in a , smooth work transition 
i 



Educational Skill Competencies f 

Conqaetencies in this area represent the basic skills and attitudes 
necessary 'for successful employment and continued learning-* Included in 
this area are the skills of communication, mathematics, use of 
information resources- and reasoning* The competencies developed in this 
area will have application in the other three competency areas as well as 
in continued learning and othep life slcills areas* 

* Th^ participant will: 

. 1* Communicate through writing a^d speaking 

o Write legibljj 

f o. 'Seiect and use job-related terms r 
v ^ 

o Complete business forms 

o Write terse factual memos ' * ' 

o Write business or employment-related letters^ 

o Write narrative reports 

o^ Take notes during informational meetings 

o Speak distinctly at appropriate volume levels 

-o Participate iti task-oriented conversations and- in group 
discussions § 

o Express one's |>oint of view regarding a job-related issue 
o Pass on taslc directions to others 

o Conduct interviews with supervisors to gain job-related* 
information 

o Use the telephone to convey information , 

2. * Understand written and oral communications 

Read business forms, notes,, memos, charts, tables and 
technical or 'reference books for their literal content 
(literal comprehension) 4 * 

o- Relate writteri information -to job requirements or problems 
(interpretive comprehension) ^ r 

o De'fine terms and abbreviations related to_jobs or employment 

o "Locate gross grammatical or procedural errors in completed 
forms*,* memos, letters o^ reports 

o Complete job-related , tasks and activities using written 
directions 

o Listen attentively & 

o Maintain eye contact with speaker 

o Gesture* appropriately while listening 

v , 
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o Paraphrase to confirm under stem ding of speakers intent 

p Distirtguish between Met and opinion in written or oral 
communication * 

o Check assumptions by askinc closed questions 
♦ , r 

p Describe pictoral, graphic or symbolic displays using words ■ 
o Complete job-related tasks following otal directibns 

Apply mathematics skills 

o Add, subtract, multiply and divide whole numbers, decimals 
and 'fractions ' . * \ 

o Perform operations in correct order when solving problems 
involving mixed operations 

Solve job-related word problems involving mixed pperations 
Read measurement scales - \ 

Measure using metric and English scales 
Verify accuracy of measurements and calculations 
Read numerical values from graphs, charts and tables 
Use measurement equivalents, ratios and proportions 
Estimate numerical quantitities (measurements, costs and time) 
Qompare numerical values 
o" Perform calculations and transactions involving money 
o Calculate amounts needed to do. practical jobs * 
o Compute averages 

o 'interpret descriptive statistics data (mean, range, mode) 

o Calculate amounts needed ttf do practical jobs (area, 'volume, a 
linear feet, etc.) 

,o Perform operations, with time 

o Operate four-function calculator 

o v Prepare simple scale drawings or informational sketches 

Use information resources 

Match resources to information needs 

b 'Locate resource materials * 

o locate experts and obtain advice 

o Obtain information from books ? 

o Use the dictionary % 

o Synthesize information * 

o Apply information to job-related problems 
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5. Apply reasoning processes 1 

o Sort objects and data ' 

o Rank objects according to criteria 

• o Rate objects using qualitative criteria " \ 
o * Develop classifications for objectives 
«, o. Seguence tasks to achieve outcomes 
o ^Prioritize tasks + 

6 Establish goals • ' 

a o Prepare plans 

o Diagnose problems 

o Identify solution alternatives 

o Evaluate solutions using criteria 

o Select alternative most likely to produce desired results 

o Set standards (quality, quantity, time) 

Occupational Skill Competencies - 

* * • * *> 

The ar«a of emplbyability competence includes knowledge and skills needed 
to perform job tasks successfully. These skills are more technical and 
specialized and relates to one or a group of ^specific occupations.' They 
are sometimes called vocational 'skills. 

It is in^ossible to develop a single set of competencies that applies to 
all occupations. It is. more usual to develop a set of competencies that 
pertain to a specific occupation—welder fox example. If 'this approach 
is taken, it is necessary to develop many sets of < competencies to meet 
diverse needs of youth and employers.^ — - 

- f 

A second approach is to -develop a more general set of cqinpetencies that 
fit STcluster of related occupations—metals for example. In this case, 
there are fewer sets pf competencies, but the copi^aetencies do not fit a . 
single occupation perfectly. § ^ % ? * 

A third approach is to develop a set of generally useful occupational 
skiils. This set would include CJompetencies useful across a wide range 
of occupations. The benefit is having to develop only a single set of r 
competencies; the problem i* that the competencies will not fit any 
occupation closely. * » 

Each local area has a choice of * approaches. A sample set of competencies 
for each of the tteee approaches has been- developed to facilitate 
decision making. • 
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Occupatidn Specific Competencies— Welder C 

The participant will demonstrate: 

*» 

1* Knowledge of metals 

o Properties of metal 
a o Structural shapes of metals 

2» Oxyacetylene welding skills 

o Safety rules and terminology 

? o Equipment use and csyce 

o Cutting with oxyacetylene equipment, straight lines t circles 
and curves/ bevels m * 

q Piercing techniques 

o Cutting round stock and pipe 

o Puddling in flat position 

k o Beading yith rod 

o Fillet welds 

• q Butt joint welds J - 

o ^langfe joint v welds 

o Lap joint welds 

o Corner joint^ welds 

o Bronze welding * / 

o Cast iron welding 

o Silver brazing 

b Soft soldering 

o Welding aluminum, stainless steel, magnesium and white metal 

* « 
3. Metal arc welding ^ 

o ' Safety rules and terminology ' . 

o * Equipment use and care 

1 o ' o Continuous bead welds 

o Wash bfead welds * 

o fillet weld v ' - 

. o Square* and vee butt Joint welds i 

o Outsi^S corner- welds ^ 



i 
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Occupational Cluster Competencies — Metals Cluster » 

Participants will demonstrate: 

1# Knowledge of the scope and distribution of various occupations 
jiin the metals industry in the local area 

o Cite local businesses that hire people with metal working 
skills 

o Describe ten occupations in the metals industry 
o Describe demand for metal workers in the local labor market 
o Describe metal industry trade and professional organizations 

2# Common sheet metal shop operations 

\ 

o Use and care of equipment, including safety 
o Sketch and make sheet metal layouts \ 
o Shape, benA and fold sheet metal 

o Join sheet metal by riveting, soldering, seaming and making 
lock joints 

* 3. Common welding operations ^ ^ 

<^ Use and care of equipment, including safety 
o Cutting ij^als \ 
' * o Joining metals by arc and oxyacetylene methods 

4* Common machine shop operations 

6 Use and care of equipment, including safety ~ x * * 

o Use and care of measurement instruments v 

o Machine round parts ' ' % ; >S$ * 

, o Machine flat mparts * ^ A 

/ • 6^ Machine threads 

5« Common foundry operations 

s 

o Use and care of equipment, including safety 
o ' Ram and pour a sand casting 

6. Use of a variety of basic metals references 

o Charts and tables „ 

o Technical information 

4 
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Generally Useful Occupational Skills 
Participants will demonstrate: 

1. Numerical skills ' 

A- 

o Calculations and transactions involving money 
o Make and use graphs, charts and tables 
o JJffk measurement equivalents, ratios, proportions 
o Estimate numerical quantities ' 
o Compare numerical values 

o Calculate ambunts needed to do practical jobs 
o Interpret statistical data- 

2. . Communication skills 

o Ebcplain, describe, demonstrate and give directions 

oiHrrite legibly 

o Speak clearly 

' o Write reports and summaries ^ 

• o Write letters, want ads and telegrams v , j 

o Understand and follow written instructions, directions and 
inf ormation ' / 

^ o Understand and follow spoken instructions, directions arid 

information / - 

o Understand and use pictorial, graphic and symfco/Lic information 

p Interact verbally with others * ¥ 

3. Manual-perceptual skills / 

o Use common tools and equipment / 
o Make and assemble useful objects ' • 

o Adjust , orepair and maintain ^common tpols and (equipment 

*\ ' I 

o Head displays and scales 

• o Make visual representations, , 



i 
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Inf ormation-proqessing and decision-making skills > 

o Remember specifics, procedures and principles that are basic 
to continued learning and that are frequently used 

o Apply concepts, principles and procedures; in circumstances 
different from those in which they were iearned 

o Analyze information and define problems 

o ° Collect and organize, data 

o Develop and evaluate alternatives 

o Choose* from am<5hg alternatives in terms of consistent criteria 

o Devise plans, new ideas and better ways of doing things 

o Implement plans and modify based on feedback 
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APPENDIX D 
STARTER LISTS OF PUBLISHED INSTRUMENTS 
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•STARTER LISTS OP PUBLISHED ^INSTRUMENTS 



Certain features of the instrument lists provided in this section are 
important to note. First of all, not all available instruments 



are 



listed. An attempt was made to present the instruments which are .widely 
•used now dr seemed 'most likely to be pf u*e in^pljoyment training. 
These instruments have all been published within the lastr ten. years and 
represent a* wide range of publishers. / ^Ke^inblusion of an instrument 
does hot imply bur endorsement for us/ in a particular situation. That 
(fecision must be made locally based cm review lusing the guidelines*/" 
presented in Chapter" Three % Another notably 'feature of. the list is that 
it contains almost no checklists or rating scales for on-the-job 
behavioral observation or product reviews. Generally, these instruments 
need to be developed specif ically «£r thdftsituition and are not available 
from publishers. One would be well-advi Steel, Vtttough, to contact other 
employment and training projects to see if theyl have instruments 
available which could be adapted for use in your situation. 

v C * \ 

Two lists of published instruments are provided.! The first one covers 
both work maturity and pre-employment competencies , since published tests' 
often do not make distinctions between these categories. This list also, 
includes instruments focused on general aptitude land Interest which are 
not linked to specific competency but may be .uieful in initial 
Assessment to gain a better understanding of the person. Conqputerized 
career information delivery systems are now beingf developed tin many 
states to hqlp people determine career interests. \ Contact your state 
Occupational Information Coordinating Cpmmittee to obtain information on 
these and various commercial systems. , They may also be very Useful in 
initial assessment. *' \ 4 \ % 



The 'second list presents knowledge tests related to basic educational 
skills. Most of the tests described' here are ones which report 
critefion-ref erencedF ratfier than norm-referenced scores. Norm-referenced 
test scores reflect hotf one examinee compares to ariother examinee or 
group. The most common types ef norm-referenced sdpres are percentiles, 
grade^ equivalent scores and stanines.* Criterion-referenced scores 
compare an examinee 1 s perfqrmance tp pyespecif ied criteria* without 
regard' to the performance of" others. w Criterion-reference a scores are 
frequently expressed as "percent of objectives^ mastered" and "percent of 
JLtems correct f or . each learning objective". "° 1 ^ v 




♦Percentiles indicate the percent of people scoring at or below a 
given score. For example, if | person were"at'thie 80tl\ percentile, 
80 percent of the people would hfcve received equal or lower scores. 
A grade equivalent score indicates a .person's performance- on a 
particular test relative to the median" performance of students at a 
giveiv grade level and month. For example, a fifth grader who 
reives -a grade equivalent score of 8.2 on a reading. test achieved'' 
the same raw score performance as the typical eighth grader in the - 
second' monfch' of eighth grade would be expected to achieve -on the same 
fifth grade test . It does not mean the fifth grader can do* eighth 
grade work. Stanines constitute a standard score scale consisting of 
nine values with arT average of five. v v 
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Criterion-referenced scores' are particularly useful for a competency- 
based training program* These scores illustrate the extent to which a 
person has acquired certain specific sjcills or knowledge. It is seldom 
of use to know h9w one person compares to^scme national group. 

Another problem with norm-referenced scotes is that they are frequently 
thought to imply certain standards. In particular, it is often assumed 
that an above average score is satisfactory, and a below average one 
unsatisfactory. By % definition, half the people tested must .be . above 
average and half, below. Whether "average" performance is satisfactory, 
below satisfactory or superior is another issue entirely. This* decision 
requires determining what specific tasks a person needs to be able to do, 
and what levels of perf ormafcce within each tisk are acceptable. 

Grade equivalent Scores present a particular problem. On many 

.standardized tests the grade equivalent score scale is constructed in / 

.such a way that a score as high as 6.0 maybe equivalent to the chance 
score.* On such^a test, results showing that an adult is reading at the 
fifth grade level only indicate that the test was too difficult. There 
is no way to know whether the person has some reading skills or was 
simply guessing in response to test questions. • 

^ Guide to Adult Functional Literacy Assesmenfe: ^Using Existing Teste d r 
(Anderson ,.1981) contains comprehensive lists of educational skills tests. ^ 
which do Include norm-referenced tests. It also includes tests which 
measure a person's 'ability to apply basic skills to everyday 
non-job-related activities (e.g., reading road signs, balancing a 
checkbook, reading a TV schedule). These non-job-related activities are 
not likely to.be included in t£e,fdur job related competency areas 
focused on in this handbook but they are sometimes taugtft in employment 
programs 

* * « ' <♦ 

Lists of assessment instruments for occupational skills are not provided 
because of the wide variety of occupations which could be covered. Most 
training use simulations because this, is the approach that will best shpw 
(as compared to paper and pencil knowledge tests) that the person has the 
skills required in the occupation. 

Comments about the quality of instruments have been drawn from The Mental 
Measurement Yearbook (Bucos,. 1978). The yearbook, published about every 
six years, contains reviews of nearly all published tests. It is an 
excellent reference for determining what published instrument exists 
which may be best .for a particular situation. 



) 



) 



♦The chancfe score is the number of items one would expect a person to 
answer qorrectly on a multiple choice test merely by guessing. 
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STARTER LIST OP PUBLISHED ASSESSMENT INSTRUMENTS : » 
PRE-EMPLOYMENT AND HORK MATURITY COMPETENCIES 



Assessment for Career Education (ACE) , 1977 , grades 3-l£^ PREP, Inc., i 
1575 Parkway Ave., Trentpn, New Jersey 086^8. 

* The* Assessment for Carter Education was developed to measure the 
outcomes and objectives identified by the U.S. Office of 
Education which have become the central concept of~career 
education across the nation. It contains multiple choicrexest 
items with scores Reported in terms of mastery of the eight 

. ' -major objectives and 26 sub-objectives of career education. 

"Self -scoring or computerized ^scoring is available. T^ree levels 
of the test exist. • 

CAreer Maturity Inventory (CMI) , 2nd edition, by John O. Cfc^tes; 
1978; Grades 6-16? CTB/McGraw-Hill, Del Monte Research Park, * 1 
Monterey,^ California 93940. 

The Career Maturity Inventory consists of an Attitude. Scale 
(available in a screening and a counseling form) and a 
Competence Test. The Attitude Sc^le contains 50 items and 
measures decisiivehess , involvement: , independence , orientation 
and compromise' in career decision making, and requires 30-40 
minutes ^to administer. , The Competence Test contains 100 items 
and assesses abilities, in self -appraisal, occupational 
information, goal selection, planning and problem -solving, and 
requires approximately two hours to "complete. Tests are 
available in hand-scorable or Aachine-scorable editions, and 
yield taw scores which cam be converted to percentile ranks or 
'standard scores using the norms provided, or local norms. 
Suggested uses for thfe* tetet include studying career development, 
screening .fo^ career maturity, assessing guidance needs, ^ 
evaluating career education, and testing' in career counseling.' 

Career Skills Assessment Program (CSAP) : 1978; Grades 9-14 and _ 
adults: t The College Board, 888 Seventh Avenue, N6w York,' New York 

10019. ~ 

■ ¥ 

. The Career Skills Assessment Program! was developed to "identify 
the skills most central to successful career development and to 
provide the means for assessing student competencies in these 
4reas. CSAP. consists of six separate self-assessment tools 
covering skills in self-evaluation and development, career 
awareness, career decision making, employment- seeking, work 
effectiveness, and personal economics. Each area has 60-70 
-questions and can be completed, in about one hour. In addition 
u to printed materials, a sound filmstrip for staff and student 
orientation is provided; Materials can be hand scored, but 
those requesting central computer scoring have available a 
combination of norm-referenced and criterion-referenced 
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interpretations. They provide class as well as individual- . 
summary reports. .Validity studies have not yet been done; 
% thus-, users must carefully review items for quality and 

consistency for cccpetenciesi being taught. The reading level 
(mainly gtade 9-11) may be too difficult for some persons. 
Also, the range of jobs discussed gges beyond those likely to. 
be involved in CETA projects. 

Program for Assessing Youth Employment Skills (PAYES) , 1979; 
individuals with low verbal skills; Cambridge Book Company, . j 
888 Seventh. Avenue, New York, New Yofk 10019. 

* PAYES is designed to assist program directors ^counselors, and 
teachers in the guidance of persons with low verbal skills as 
they prepare for entry level employment". It is based on 

"extensive research in measurement by ETS for school dropouts, 
potential dropouts and disadvantaged youth. The batfcery is 
divided into three general categories: 'Attitudinal Measures 
\ (j6b holding skills, attitude toward supervision, 
self-confidence); Cognitive Measures dob knowledge, 
job-seeking skills, practical reasoning); and a Vocational 
Interest Inventory. Total testing time is approximately 75 
minutes. Measures of validity and reliability are presented in 
the technical manual. 

Strong-Caftpbell Interest Inventory , revised edition, by Edward K. ^ 
Strong and David P^Campbell; 1977; Ages 16 and over; Stanford 
University Press, Stanford, California 94305. 

The Strong-Campbell Interest Inventory is designed to help , 
individuals discover interests in the following occupational 
themes: investigative, artistic, realistic, conventional, 
social, and enterprising* Basic interest scales are available 
for, the following: adventi^re, agriculture, art, athlgtics, 
: business management, domestic arts; law/politics, mathematics) 
mechanical activities, medical science, medical service, 
merchandising, military activities, music/dramatics, nature, 
office practices, public speaking, religious activities, sales, 

^ science, ^social service, teaching, and writing. " Individuals # 
are' also ratfcd as to how their interests compare with 
individuals already working in 124 different occupations. An 
acadenffc orientation scale and introversion-extroversion scale 

^ ; are also included. jThe test requires from 20-3Q. minutes to 
complete, and scoring must > be purchased through authorized 
scoring agencies. 
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6. USES General Aptitude Test Battery (GATB) forms A, B, C, and D: 
1970; Grades 9-12 a nd' adults; United States Department of Labor, 
Manpower Administration, Washington, p. c. ^20210. 9 

GATB is a multiple aptitude test. battery tesiing the following 
i areas: Intelligence^ verbal, numerical, spatial, 1 form 

perception, clerical perception, motor coordinatibn, finger . 
dexterity, and majiual "dexterity . Administration time is 
approximately two and one-half hours for the total battery. * 
Also available is the USES Nonreading Aptitude Test Batfcery , a 
comparable ^instrument for nonreading populations. GATB was' 
developed by the United States Employment Service foa: use in 
its occupational counseling program. Minimum aptitude Scores 
and norms^for^ specific occupations are provided for us^ in 
counseling. Use of the GATB as a measure of intelligence is 
not advised. 



S 



7. ; Vocational Opinion Index (VOI) ,. Forms A and B; Associates for 

Research .in Behavior, Inc. /The Science Center, 34€h and M$prket ( 
/Street,, Philadelphia, %|nnsylvania 19fIlm.V ^ u 

- ..JThe Vocational Opinion Index is a snort paper-and-petfcil test 
to measure fcn individual's Job Readiness Posture (JRP) - 
attitudes, perceptions and' motivations which impact on ability 
to obtain and maintain a job. The 58 item VOI can be » ^ * 

administered in about 20 minutes and assesses the following 
dimensions: attractions of work, losses associated wit)r^ * 
obtaining and maintaining a job, and barriers to eiBj^oyment* J , 
The twrt provides a diagnosis of reasons contributing to an * 
j individual's classification as a potential nonworker, and can 

be used to develop prescriptive material to improve the JRP. . ? — J 
The test, available in English and Spanish, must be, 'scored by . ~ ' 

the publisher. The VOI was designed, Ibested and normed , , 
specifically for the ecpnomically disadvantaged, population. 
Research has been done to establish the validity and 
* reliability of VOI. w 

' N - 

8* Wide /Range Smployability Saigple Test (WREST) by Joseph F. Jastak and* * * ' 
Sarah Jastak; 1980; Ages'16-55j Jastak Associates, Inc., 
1526 Gilpin Avenue, Wilmington, Delawa^d 1980S. ( v 

The WREST was originally designed, for use with mentally and , • 
physically handicapped individuals/ but has more recently, been 
revised for use with the general population. The ten work 

t sample tasks included are: fplding, stapling/ packaging, 
measuring,' tag stringing, • gluing, collating, color and shade 
matcj^lng, pattern matching, and, assembling.' Each task takes 
.sevpn^to 15 minutes to complete, while tfce entire test requires 
about one and one-half , hours _f or individual administration, 'and 

. , two hours for small .groups of three to six persons* Norms are 
provided for production quantity and production quality for the 
general population and for special groups. 



\ 
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ASSESSMENT INSTRUMENTS: 
' EDUCATIONAL SKILLS* 



Multisubject Batteries (usually a combination of reading, writing, 
language arts and/or mathematics) 

1# frdult Basi c Learning Examination y(ABLS) , Forms A and B, by Bjom 
Karlsen, Richard Madden, and Eric P. Gardner; 1967-74; Adults with 
achievement levels grades 1-12; Psychological Corporation, 
304 E. 45th Street, New York, New York 10017. 

°ABLE is a norm-referenced test with subscores available in the 4 ' 
following areas: vocabulary, reading, spelling, and 
arithmetic. The tests fire available for adults with skills at 
the following grade levels: l-4^LevTT 1 )7~S^~(Leve^"^ , and~ 
9-12 (Level 3). SelectABLE is available for use as a 
preliminary ^screening test to determine the appropriate level 
of ABLE for each adult. Test administration time for levels 1 
and.; 2 is 145 'minutes, and for level 3, 250 minutes. All levels 
have been validated and reliabilities established with 
heterogeneous" youth and adults enrolled *in schools or 
community-base£ training programs. 

2 * Adult Performance Level Survey (APLS) ; 1976; n£yh school and adults; 
American College Testing Program, P.O. Box 168, Iowa City, Iowa 
•52240. 

• 

*■ ^The APL is a* criterion-referenced measure of functional 

literacy. Scores are available for the following .skills : 
identification of facts and terms, reading, I writing, 
computation and problem solving/ Content scores are also 

^ - computed on the following: community resources, occupational 

. ( * knowledge, consumer economics, health, and government *and law* 
Two levels are available, one for high school students, and one 
for rfatxlts, in either machine or hand* sporable editionls. 
/ Approximate administration time is 60 minutes for the high 

school level, and 75 minutes for £he adult leve\. Norms, 
standard deviations, ajad below, average, and .above ratings are 
given for individuals, as well as groups.. Supplementary 
directions are given for interpreting the data, and for 
modifying instruction. 



♦Most of the instruments included in this afectidn are criterion- 
referenced tests. If norm-referenced tests are desired, consult the 
appendix of Guidelines for Selecting Basic Skills and Life Skillq Tests 
(see reference 1-iat). ' ~ ' 



Analysis of Skills: language Arts (ASK-Language Arts) by 

O. p. Ander halter, Neil J. Vail, and Joseph F. Papenfuss; 1975-76; 

Grades 2-8; Scholastic Testing, Service, Inc., . 

480 Meyer Road, Bensenville, Illinois 60106. 

ASK-Language Arts is an objective-referenced and 
nonn-referenced test measuring skills in capitalization and 
punctuation, usage, and sentence knowledge/composing process. 
Each of 36-58 skills are tested by three items^ and the results 
are given as mastery, partial mastery and non-mastery of each s 
skill. The test is available in six levels, with the 
overlapping of grades (2-3, 3-4, etc.)* Approximate 
administration times vary from 60 to 90 minutes. , Rental and 
scoring services are available.. No data on reliability is 
presented, nor are norms giveh for the^kill scores. 

Diagnostic Skills Battery (DSB) , Form A and B; 1977; Grades 1-8; 
S cho l ast i c Tes ting .Service, 480 Meyer Road, * Bensenville, Illinois 60106. 



The Diagnostic Skills Battery tests reading and mathematics in' 
four levels for grades 1-8 (Levels 12, 34, 56, and 78), with 
language arts also included in Levels 34, 56, and 78. The 
tests are similar to thdfce of the Analysis of Skills test 
series (published by the same company as described below), but 
test fewer objectives. DSB focuses on 35-45 objectives in each 
area at each level, using two items per objective. Test 
administration time is approximately two hours, 40 minutes. 
Both objective-referenced scores and * norm- referenced scores are 
reported. The General Manual contains the rationale for the 
tests as well as an explanation of the scores reported, 
descriptions of the standardization procedures, and internal 
characteristics of the test are also provided. 

General Educational Performance Index , Forms AA and BB, by 

Don P. Seaman and Anna C. Seaman; 1981; Adults; Steck-Vaughn 
Company, g?.0. Box 2028, Austin, Texas 78768. 

The General Educational Performance Index was devised to 
} measure the extjint to which adults are ready to attempt the 
General Educational Development test. The five subtests 
include writfng, reading, mathematics, social studies, -and 
sciehce. Administratipn time is approximately three hours. 
Test scores may be useful for grouping for instructional 
purposes or follqwing'progress in test-retest situations. A 
description of the test construction and standardization 
procedures, along with the test's validity and reliability, are 
included in the Manual of Directions. 
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High School Equivalency Test Guide by Raymond G. Hodges; 1975; 
Adults P. A.R. Inc., Abbott Park Place, Providence, Rhode Island. 

The Hifrh School Equivalency Test Guide is attest primer for the 
GED, similar in some aspects to the General Education 
Performance Index described earlier. Subscores are available 
in the following areas: correctness and effectiveness off 
expression (spelling, capitalization, punctuation, usage, r 
♦effectiveness' of expression, diction and style); interpretation 
of reading materials in literature, social studies and general, 
science; and general mathematical Ability. The test guide 
contains 310 multiple choice items. 

Minimum Essentials Tqst , Forms A, B, and C, by John W. Wick and 
Jeffrey K. Smith; .1980; Grades 8-12 and adults; Scott, Foresman and 
Company, 1900 East Lake Ave., Glenview, Illinois 60025. 

The Minimum Essentials Test measures student achievement in 
reading, language, and mathematics, ancLprovides information on 
the student's ability to apply basic skills to life 
situations. An optional writing .test is included. The 
124-item test requires about one and one-half hours to 
administer. The reading section covers literal comprehension , 
inferential comprehension, context* clues, and main idea. The 
language section assesses knowledge in punctuation, 
capitalization, and sentence structure. The mathematics test 
covers basic arithmetic operations for whole numbers, 
fractions, decimals and percents. The life skills section 
tests the application of those skills in communication; 
finance; government and law; health Safety and nutrition; 
transportation; and occupations. Both norm-referenced and 
criterion-referenced scoring information is available", with 
standards of mastery set by the local user. Information on the 
tests' development and additional technical data are provided 
in the Teacher's Manual. , 

Objectives -Referenced Bank of Items an<TTests; Reading and 
Communication Skills (ORBIT; RCS) ; 1975; Grades K- Adults; / 
CTB/McGraw-Hill, Del Monte Research Park, Monterey, California 93940. 

ORBIT; RCS is a collection of 335 objectives and Corresponding 
* test items from which educators can selefct those most 

appropriate for their local goals and programs. Objectives and 
items are available in the following areas; visual 
discrimination, phonic analysis, structural analysis, word 
meaning, literal comprehension, reference skills, language 
mechanics and language expression. Each objective indicates 
the vocabulary level of the corresponding items (primary 
through 10th grade). Scores reported for each subtest with 
the criterion level for all subtests established at 75 
percent.' Each subtest requires approximately five minutes to 
administer. If a set of items requires oral administration, or 
requires more than one page, that information is noted in the 
catalog of objectives. Tests are available in two formats, and 
include an examiner's manual. Objectives booklets are also 

avai lable. Low reli abilit ies for su btests, a n d th e need for 

precise recordkeeping, may be concerns for the potential user. 



9* SRA Norm-Referenced/Critgrion-Referenced Testing Program ; 1971-76; 
Grades 3-9; Science Research Associates, Inc'. , 155 ty* Wacker Drive, 
Chicago, Illinoi's 60606. 

* 

The SRA Norm-Referenced/Criterion-Referenced Testing Program * is 
a collection^of tests that can be /used in a variety of • 
combinations; to fit local needs and circumstances. The 
norm-referenced tests contain 30 items and re<juire 25 minutes 
to administer. The criterion-referenced" tests require 45 
minutes to administer and contain 13-15f\£>bjectives with three 
items per objective. Reading CRT's are. available in the 
following areas: phonic analysis,' structural analysis, 
vocabulary, comprehension and study skills. Agencies desiring 
customized 'CRTs may select from the Mastery; An Evaluation 
Tool reading or mathematics objectives. The Survival Skills 
Reading and Mathematics tests are presented as ^an optional 
component. Some technical, data is Available in the NRT/CRT 
Interpretive Manual, but close examination should be made of % 
the validity and reliability data of the individual tests being 
considered. - • 

10. SRA Survival Skills in Reading and Mathematics ; 1976; Grades ^t? 
, 6-Adult; Science Research Associates, Inc., 155 N. Wacker Dr., 
Chicago, Illinois 60606. 



The SRA Survival Skills Test , a criterion-referenced test, 
measures 20 objectives in math and 20 objectives in reading. 
Each objective is measured uSing three items. The test 
— requires approximately two hours to complete. Tests may be 

hand or machine scored. Suggested mastery level for each item 
is 100 percent while mastery level for each test is suggested 
as 80 percent:. The available student data includes the percent 
of students mastering each objective, performance on each item, 
and overall test pyrf ormande.- Group data includes tfce percent 
of students mastering each objective, the percent of students* 
responding to jgagh item correctly, the average overall te$t 
performance, and the distribution of students mastering 
^cumuljative numbers of objectives. > 

11 • Steck-Vaughn Placement Survey for Adult Basic Education , Forms I-K, 
I-L, II-M, and II-Q, by Beth Ann Phillips; Adults with educational 
skills from grades 1-8; Steck-Vaughn Company, P.O. Box 2028, Austin, 
Texas 78768. 



The Steck-Vaughn Placement Survey for Adult Basic Education is 
designed to Assess skills from a nonreading level through 8th 
grade* Skills are assessed in the areas of reading, language, 
and mathematics and include phonics,* vocabularyT 7 comprehension, w 
capitalisation, punctuation, spelling, English usage, money , 
word fctoblems, and operations with whole -numbers. Test -4 
administration time is approximately 15 minutes for each 0 ; 
section, or 45 minutes for the entire sux^ey. A % wcrrd list is 
used as an initial screening device to prevent, learners from 
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experiencing excessive frustration when taking certain pa£ts of 
this survey. A Skills Breakdown Chart is provided to help 
identify specific strengths and weaknesses* Prescriptions are 
given by grade level, keyed to other Steck-Vaughn 
publications. Technical data is not included in the examiner's 
manual. 

.12. Tests of Adult Basic Education (TABE) ; 1976; Adults with reading 
levels grade 2-9; CTB/McGraw-HiLl, Del Monte -Research Park, 
Monterey, California 9394CT. 

TABE is a/ measure of adult proficiency in the basic skills of 
reading, mathematics and language. The test is available in 
three levels, E (easy), M (medium), and D (difficult), and 
requires from 127 minutes to 209 minutes to complete. SubtesJ: - 
scores are reported for vocabulary, comprehension, computation, 
concepts and problems, mechanics and expression, and spelling. 
Scores are reported as standard scores and grade^ecpiivalent 
scores. TABE is the adult version of the California 
Achievement Test (CAT-70) , and uses the s same basic content and 
' * format. Norms' are available, but they are based on CAT-70 , 

rather than TABE . # 

13. USES Basic Occupational Literacy Test (BOLT) , Fbrm% A, B, and C; 

1973; Educcat^onally disadvantaged adults; United States Department 
of Labor, Bureau of Labor Statistics, 1515 Broadway, New York, 
New York 10036. 

BOLT is a test of basic reading and math skills developed for 
use with educationally disadvantaged adults. It is available 
in three forms at four levels': fundamental, basic 
intermediate, high intermediate, and" advanced. A(jpinistration 
of the BOLT is preceded by giving the Wide Range Scale , a brief 
screening test in reading and math, to /determine the 
appropriate level of BOLT for ^he individual. Subscale scores 
are available in reading vocabulary, reading comprehension, 
arithmetic computation, and arithmetic readtoning. Subtest raw 
scores e^re converted to standard scores, which are translated 
to a GED^General Educational Development) level. Counselors 
can compare literacy skills on the subtests with occupational 
skill requirements provided in the supplements to facilitate 
vocational placement. The entire test requires approximately 
130 minutes to administer, and can be hand or machine scored. 
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STARTER LIST OF PUBLISHED 
1 ASSESSMENT INSTRUMENTS: 
. READING TESTS 



Analysis of Skills: Reading (ASK-Reading) by O.F. Anderhalter and 
Frances Shands; 1974-76; Grades 1-8; Scholastic Testing Service, 
Inc., 480 Meyer Road, Berisenville, Illinois 60106. 

. ASK: ^Reading is a criterion- and norm-referenced test 
measuring skills in the following areas:' word analysis, 
comprehension, an& study skills. Skill scores are interpreted 
as mastery, partial mastery, and non-mastery in the following 
categories: discrimination, phonetic analysis, structural 
analysis,^ ybrd recognition, vocabulary in context, literal 
comprehension, infer ential c omprehension , critical 
comprehension, 'library and reference skills, organization 
skills, pictorial and graphic material, and following^ 
directions. In most cases, each ski^l is measured by three 
items. Total test administration times vary (depending on the 
test level) from approximately 135 to 175 minutes, divided 
among three 'testing sessions. 



iterion-Referenced Tests: Reading Tactics , Levels A - F, 

Olive Stafford Niles and J. Japp Tuinman; 1981; Grades 7-12; 
Scott-, Foresman and Company, 1900 East Lake Avenue, Glenview, 

„ IL 60025. . ) • 

it i 

Reading. Tactics is an instructional program which is^designed 
pr imarily for students who are typically below average in their 
reading skills. The criterion-referenced tests are the 
diagnostic and evaluative compliments of this system. The 
concepts *nd skills taught in the Tactics program are defined 
„ by 12 Program Objectives and 78 Skills Objectives in the 

following areas: word attack (context, structure, sound, and 
dictionary), inferences, central focus, Vela t ions hips, sentence 
meaning, judgments, figurative language, imagery, and 
flexibility. Each objective is measured by a siaF- to ei^ht- " 
item subtest, which is available in either hand scored or 
machine* scored editions. These tests can be used to their 
greatest advantage when used in conjunction with the other 
elements of the instructional program. 
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3* Fountain Valley Teacher Support System an Secondary Reading, ^ 

(EVTSS-SR) ; 1976; Grades 7-12;. Richard L. Zyeig Associates, Inc., . 
* 20800 Beadh Blvd. , Huntington Beach, California 92648. ' 

The Fountain Valley teacher Support System in Secondary heading 
(FVTSS-SR) is a. continuation of the earlier edition which was 
.developed for grades 1-6. Sixty-one specific objectives are -* 
* — divided into three subtest areas: comprehension, study skills, 

and vocabulary. Subtests are of varying lengths with the 
majority consisting of four or six items*. An optional survey 
< ' test is available for initial screening of students. While the 

^ system assesdps student mastefy of specific objectives and 

provides references for remediation, the extensive record- 
keeping required may inhibit v some potential users. 

— Gray Oral^^ing Test ; 1967; Grades l-Mult;^obbs-M6rrill -Company-. 

4300 West 62nd Street, Indianapolis, Indiana 46268. 

rhe Gray Oral Reading Test is useful as an adjunct to silent 
reading tests. The test is available in four comparable fonna, * 
with 13 passages ranging in difficulty levels y from pre-primer >w 
through adult. The examiner presents differ ent pas sages for * 
the student to read aloud, progressing until the student makes 
seven or more errors on two successive passages. Errors are 
/ % 9 classified as aid, gross mispronunciation of a word', partial 

mispronunciation, omission of a word or group of words, > 
insertion of a word or group of words, repetition of one or 
. more words, and inverting or changing worp order, instructions 
are provided for scoring the four comprehension items given at 
the conclusion of each passage. 

5# Individualized Criterion Referenced Testing: Reading (ICRTR) , Forms 
A arfd B; 1973-76; Grades K-8; Educational Development Corporation, 
P.O. Box 45663, Tulsa, OK ?4145. * o 

The ICRTR consists of nine levels, of tests coveriife 345 
overlapping objectives, with two items per objective. Each 
level includes from, two to nine minibooklets covering from 16 
to 72 objectives. Objectives include "letter recognition, : 
phonetic analysis, structural analysis, literal and critical, 
, comprehension, and vocabulary. Tests are available in machine- 
and hand-scorable editions. The items appear to have content 
validity in that they reflect the objectives accurately. The. — 
manual does not discuss in detail other questions of' objective 
and item selection, nor $oes i!t discuss field te^t information 
on final test fetalis, assignment of objectives to a level, 
development of the continuum of objectives, and the 'reliability 
of the test in general. ^ Test results are reported in the form 
of a student summary, listing the objectives the student has 
mastered, as well as the objectives needing review. 
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/ 6. An Evaluation Tool: Reading (System for Ofrjective-Jtesed 

Assessment—Reading (SOBAR) , Fdrms L and M by the Center for the 
Study, of Evaluation, University of Calif oniia^ at I&s Angeles; 
1974-76; Grades K-9; Science Research Associates, Inc., 155 N. 
Wacker Drive, Chicago, Illinois 60606. . s 

SOBAR is a set of ctiterion-ref erenced subtests in klpc areas: 
comprehension, letter recognition, phonic analysis, structural < 
analysis, study skills and vocabulary. Each grade level test * 
> includes from 25-35 objectives, each meassured by three items. 
Each subtest can be administered in approximately three 
minutes. The potential user has the option of selecting from 
302 objectives for a customized test in either English or 
- Spanish. Scoring s ervices av ailable include performance on 
each objective, as well as ^performance on the total test for 
f * „ each pupil, class or system. The manual recommends a program 

of criterion-referenced probes called Diagnosis: An 
* Instructional' Aid . SOBAR provides additional instructional > 

prescriptions based pn pages from basal texts and supplementary 
materials. 

7 * Performance Assessment in»Reading (PAIR) ; 1978; Grades 7-9; 

CTB/MeGraw-Hil 1 , Del Monte Research Park; Monterey, California 93940. 

PAIR provides identification, of learner needs in basic reading 
skills and instructional prescriptions referenced to those 
skillq. The 72-item test is divided into two sections for 4| 
administration: reading enabling skills and life role 
u m applications. Each section requires approximately 50 minutes 

f + to, complete*. Reading subtests include vocabulary, literal 

v comprehension, critical comprehension, and location/study 

skilli. Life role applications include pictorial 
representation, reference, forms and documents and^rules and 
regulations. Local programs set the standards of mastery. 4 
Reporting services are available from the 'publish^?. 

8. Power, Reading Survey' Test , by William E. Blanton, James L. Laffey, ^ 
Edward L. Robbins, and Carl B. Smith; 1973-75; Grades 1-12; .BFA 
Educational Media, 2211 Michigan Avenue, P.O. Box 1795 Santa Moriica, 

* California 90406. 

The Power Reading Survey T6st is a series of three 
criterion-referenced tests measuring 105 objectives in three 

* areas: word recognition, comprehension and study skills. Test 
administration time ranges from one. £o five hours depending on 
the test level used. Approximate test administration times are 

41 one hour for Power l,^«r6 hours for Power 2, and five hours for 
Power 3. The Power tests are a part of the Power Reading r 
System, -a program for diagnosis and remedial instruction which 
includes lesson plans and additional tests which must be 
reproduced locally. Sccte of the tests in each level must be 
administered individually. A recordkeeping systenuis included 
for monitoring the progress of individual students by* objective* 
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Stanford Diagnostic Reading Test' (SDRT) , Poras A and B, by Bjonv 
Karlsen, Richard Madden, and Eric P. Gardner; 1966-76; Grades 
1.5-13; The Psychological Corporation^ 757 Third Avenue, New York, 
New York 10017. # . * 

The Stanford Diagnostic Reading Test is available in macihine- 
or hand-scorable editions. at four levels. The red level, 
designed for grades one-three tests abilities in word reading, 
comprehension, auditory vocabulary, .auditory discrimination and* 
phonetic analysis. The green level, for grades two-five, 
measures auditory vocabulary, auditory discrimination, phonetic, 
analysis, structural analysis, and iiteral and inferential * 
comprehension.' The brown level, for people with reading 
ability in the four-nine grade ra/ige assesses abilities in 
auditory vocabulary, literal and inferential comprehension, 
phonetic analysis, structural analysis, and Reading rate. 
— blue-level (^Trades nine-13^-tes ts lite ra l and infe rerrety 
„ comprehension, vocabulary (word meaning and word parts), 
phonetic analysis, structural analysis, and rate (scanning, 
skimming and fast reading). Administration time£ vary from 115 
minutes to 165 minutes. Content-referenced scores and norm- 
referenced scores are provided along yith suggestions for 
. prescriptive teaching* and instructional alternatives. 

Woodcock Reading Mastery Tests . Forms A and B; by Richard W. 
Woodcock* 1972-73; Grades K-12; American^Guidance Service, Inc., 
Publisher's Building, Circle Pines, Minnesota 55014. 

The Woodcock Reading Mastery ^ests is an individually 
administered test measuring abilities in the following areas: 
'letter idelnti^f ication, word identification, word comprehension, 
and passage con^rehension. Although the predicted 
administration^ time is 20-30 minutes, poor readers may require 
much longer. — 
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STARTER LIST OF POT&JSHED 
ASSESSMENT INSTRUMENTS: V 
MATHEMATICS TESTS 



. Analysis of Skillq/: Mathematics (ASK-Mathematics) by O.F. 
Anderhalter; 1974^76; Grades 1-8; Scholastic Testing -Service , 3jic., 
480 Meyer Road, Sensenvilie, Illinois 60106. 

ASK-Mathematics is an objective-ref erenced- and ndra-referenced 
test measuring skills in the following areas: computational 
Skills, concepts, and application* . Skill scores are provided 
for the following areas: positive whole numbers, fractions, 
, • and decimal's; negative integers; computational processes; 

number s-and-numerat±tin systems , -pr # Og«rtiesr^and .theory; "set 
notation and operation; common measures; metric and nonmetrlc 
geometry; percents; statistics and probability; trigonometry; ■ 
functions and graphs; .ratios and proportions; mathematical ' 
sentences and algebra; everyday problems; and use of specific 
• concepts. Each skill is tested by^ three items, and th4*results 
are given as mastery; partial mastery, and nonmastery of each 
skilly The test is available in seven levels, with overlapping 
of the' grades 1-8 (1-2, 2-3, etc.). Approximate administration 
times vary from 100-180 minutes, divided among three testing 
sessions. . ' 

• ; , - 

diagnostic Mathematics Inventory (DMI) Levels A - G, by John 
Gessel; 1971-75; Grades 1*5-8.5; CTB/McGraw^£ll # Del Monte Research 
Park, Monterey, California 93940. f 

v - The DMI is a criterion-referenced test measuring 325^ specific 
p objectives using four items per objective. The individual' 
diagnostic report and the objectives mastery report for the 
entire class reflect .learning on an pbjectiVe-by-objective 
basis. The objectives are in th6 following general areasYsJ) 
numeration, computation, fractions, decimals-, negative numbers, 
number properties, numerical ^reasoning, number theory, metric 
geometry, measurement, mpney, temperature, time*, geometry, 
pe?c§nts, and set notation. Suggested mastery level is 75 
percent though local standards may be substituted. Included in 
the testing materials are a teacher's guide and a leamihg 
activities guide. Also available are reference guides to each 
. . textbook keyed to DMI^ and a guide to nontextbook materials. 
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Fountain" Valley Teacher Support v System in Mathematics (FVTSS-M)' ; 
1972-74; Grades K-8; Richard L. Zweig Associates, Inc., 20800 Beach 

Blvd.^_Hun^^ 

• •* 

7116 Fountain V alley Teacher Support System in Mathematics is a 
series of 786 subtests, each measuring a specific objective in 
one of the following areas: applications, functions c andV 
graphs, geometry, logical thinking, measurement, numbers and 3 
- • operations, problem solving, sets, statistics *ndj?t£bability. 

The test is administered by tape cassette in bot^se^f^scoring 
^ and hand-scoring editions . Tests require f roBr 1L-25 ,mjriutes 
per subtest, Looseleaf prescription guides and individual 
?* cor <* fo^s are included with the test manual for each grade. 
^ While, this system assesses student mastery of., specif ic 

objectives and provides rftf ernnfeeg.. fpr^ Mediation , ' the amount— 
of recordkeeping required may be pro^bitive for- seme 
instructors. - ^W"- 

^• 4 " Individual Pupil Monitoring jSys tern—Mathematics (IPMS-MathematicsJ , 
Forms S and T; 1973; Gra<{M 1-8; Riverside Publishing Company, 1919 
South Highland Avenue, Lombard, Illinois 60148. 

> 

The IPMS-Mathemat ic s is a criterion-referenced test measuring 
objectives in the following areas: numeration and number 
systems, basic mathematical operations, geometry, measurement,' 
problem solving, probability and statistics, and sets. * ^he 
system includes a series attests that are grouped into \ 
"assessment modules", ansjHfegheets, pupil progress records^ 

•teache* management records; -behavioral objective booklets, 
reference bjpoklets for individualizing instruction,, and / 
^ teachers 1 guides. There are from 47-64 objectives for each * 
grade level, with ^2 objectives in all. The objectives in the 
assessment" module match those generally taught in the fall, 

-winter or spring quarter of each year. Levels one-three have 
five items per objective, while levels four-eight ^iave ten 
ft items per objective. ^ 
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5 - Individualized Criterion Referenced Testing; Math (ICRTM) , Forms A 
and B; 1973-77; Grades 1-8; Educational Development Corporation, 
P.O. Box 45663, Tulsa, Oklahoma 74145. 

* / 
The ICRTM consists of eight levels of test s^ measuring 312 
objectives in the following areas: sets; bases/numeration 
systems; addition and subtraction/whole numbers; place/ value; 
equations and inequalities; properties of addition and 
subtraction; money; time; geometry; fraction^; measurement; 
jrord problems; multiplication of whole numbers; division of 
whole numbers; properties of multiplication and ^division; 
multiples/prime and coo^osition, factorization; estimation and 
rounding; graphs and *sCale drawings; decimals and percent s; 
exponential notation; ratio and proportion; and opeijations of 
rational numbers/integers. Objectives are measured by two 
items each, and are grouped in four or five booklets for each 
lavel. Mastery levels are established as 100 percent for all 
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tests • The tests are available in machine- or hand-scorable 
editions. Test results are reported as a student summary, 
listing the objectives the student has master*^ as well as' the 
objectives needing review. _ 

1 / • 

KteyMath Diagnostic Arithmetic Test by Austin J. Connolly, William 
Nachtman, and E. Milo Pritchett; v 1971-76; K-6; American Guidance 
Service, inc., Publisher's Building, Circle Pines, Minnesota 55014. 

7" * • - ' 

KeyMath Diagnostic Arithmetic Test is an individually 
administered test originally developed for use with educable 
mentally Retarded children. Subtest scores are available in 

\. t* 1 ® following areas: numeration, fractions, geometry, symbols, 
addition, subtraction, multiplication, division, mental 

— ^-Qgrnputati o n ,. ftumerlcal reasoning, word problems, missing 



elements, money, measurement, time and metrics. The items are 
keyed to objective^|nd deficit areas are defined in detail, 
allowing accurate-remediation. The items are presented in an 
easel kit folder and* the test requires approximately 30 minutes 
to administer. The* testing packet includes diagnostic re9ords 
and manual. 




Mastery: An Evaluation Tool: Mathematics , Eorms X and Y;-jZrades 

K-£; Science Research Associates, Inc., 155 N. Wacker* 

Chicago Illinois 60606. / 

• / * 

^ Mastery: An Evaluation/Tool: Mathematics is a set of 

criterion-referenced subtests in ten areas—fractional numbers; 
( geometry j integers; measuriement ; nunflbers and numerals; sets, 
functions At graphing; sets, logical thinking and geometry; 
statisticsTffla probability; whole number confutations; and 
whole numbers. Each grade* level test includes from 15 to 40 
*v objectives, each measured by three items. They can be 

administered in approximately three minutes each. Scoring and 
reporting are similar to SpBAR with mastery levels at 
100 percent for each subtest. A U9er*s guide, manual, and 
manual supplement are also available, 

Mathematics: IPX Objectives-Based* Tegts , Forms A and B; by Ira 
Moskow, Denis Purcell, and Don May; 1973-76; Grades K-9; 
Instructional Objeptlve.s Exchange, Box 24095, Los Angeles, 
California 90024^ , , 

Mathematics; IPX Objectives-Based Testg are 280 ,* * 
criterion-referenced tests ^ith five-ten items based on 
behavioral objectives. The K-6 tests assess skills in the 
^following areas: iets and numbers,, operations and properties, 
numerations and relations/ measurement and geometry. Tests for 
grades seven-nine include measures in the following areas: 
elements, symbolism, measurement, geometry, operations and 
relations. Printed on spirit masters, each test requires 
five-tei> minutes to administer. 
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3* Objectives-Referenced Bank of Items and Tests: Mathematics 

(ORBIT; M); 1975; Grades K-Adults; CTB/McGraw^Hill ^ Del Monte 
Research, Park, Morifcprey, California^ 93940* 

ORBIT; M is a customized, criterion-referenced test consisting 
of four-item, single objective subtests covering up to 50 
objectives. The 507 available objectives correspond with items 
in the following 18 areas: addition, subtraction, 
multiplication, division, number and numeration, number theory, 
number sentences, number properties, set theory, common scales, 
geometry, measurement, graphs, coordinate geometry, word 
problems, probability and statistics, algebra, ratio arid 
proportion and percent. Subtests are categorized according td 
the grade range~ in which each objective is typically introduced 
and mastered. Each subtest requires approximately five minutes 
to administer. If . a subtest requires oral administration/ or 
requires more than o^e fcage, it is noted in the catalog of 
objectives Available from the publisher. The customized tests— 
are available in two formats, hand-scorable, o^ machine- 
\scorable, and each includes an examiner's manual. 

10. Stanford Diagnostic Mathematics Test is available in machine- or 

hand-sborable editions at four levels. The red level, designed for 
grades one-fo\jr, tests abilities in number system and 
numeration-simple and compound, additipn and subtraction, story 
problems, tables ar^L graphs, geometric shapes and properties, time 
. and^jnjoney. The.gteen level, for grades three-six, tests concepts of 
whole mafibers, decimals, ordering/' rounding, addition, subtraction, 
multiplication, division, simple and compound number sentences, 
story problems, geometric shapes and properties, metric system, time 
and morjey. The brown level (grades five-eight) tests the same * 
concepts as the previous one, with adjustments made for ^grade 
level. It also includes both metric and English systems of 
measure. The blue level, (grades seVen-13) measures many of the 
same concepts as the previous two levels, with the addition 'of rate' 
problems. Test administration times -range from 95-120 minutes per 
level. Norms are provided, along with percentile^ranks and stanines 
by grade. Individual diagnostic reports and pupil^Jrof iles identify 
individual strengths and weaknesses and facilitate remediation in 
the general areas. If more detailed information is needed, a 
locally produced test might serve better. m 
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PUBLISHERS 1 NAMES Atffr ADDRESSES 



Listed bplow are/assessment instrument publishers mentioned in the T 
Handbook as well as other publishers whom readets may wish to Contact for 
possible a^festaent instruments. 

Academic Therapy Publications " 
20 Commercial plvd. 
Novato, California 94947. 

Addison-Wesley Publishing Co., Inc. . s 

South Street 

Reading, Massachusetts 0186> 

•American College Testing Program 
P.O. Box 168 

' Iowa City, Iowa 53340 • ' 

American Guidance* Service 7 . % 

Publishers Building > 
Circle Pines/ Minnesota 50014 

, Associates for Reseawh in ' ' L 

Behavior, Inc. 
The Science Center s 
34th and Market Street, 

Philadelphia, Pennsylvania 190,04 9 m 
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«*A Educational Media *~ 
2211 Michigan Avenue 
P.O. Box 1795 

Santa Monica, California 90406 

Bobbs -Merrill Col, Inc. 
4300 West 62nd Street 
* Indianapolis , Indiana 46268 
P 

( Brador Publications, Inc. 
Livonia, Mew York 14487 

Bureau of Educational Mwsutement 
Etopo&a State University^ 

Emporia, Kansas 66801 ^ . 1 

CAL Press, Inc. 7 — 

76 Madison - Avenue 

Hew York, Mew York 10016 £ 

California State JDepartment 

of. Education 
721 Capital Mall * ' 

Sacramento, California 95814 ' 

0 . 

; d-2o 136 s - 



California Tasting Bureau/ 
- , McGraw-Hill 

' ^ Dal Honte Research gPark _ 

T , r Hohtferey, California 93940 ^ 

«. 

Cambridge Book Company 

888 Seventh Avenue 

Hew Yorjc, New York 10019 

The Center for Applied Research in x 
Sducatipn , Inc. 
J flpute 59 * p 

jT l&st Nyack, New York 10994 

I 

r Charles B. Merrill Publishing Company 
/ ]1300 Alum Creek Drive 
^Columbus, Ohio 43216 

The College Board 
888 Seventh Avenue 
■ New York, New York 10019 

J Cutronics Educational Institute 5 * 

128 W. 56th Street 
^Bayonne, New Jersey 07002 



Educational Development Corporation 

• P.O. Box 45663 
Tulsa, Oklahoma 74145 

* Educational Testing Service 
^— : — Princ e to n: / New J e r se y 0854 1 

Educators Publishing Service 
•75 Mdulton Street 

Cambridge, Massachusetts 0213.8 

• * * 

Barcourt, Brace* Jovanovich, 

Publishers r 
757 Thi/d Avenue 
\ New York, New' York 10017 

o 

» Instructional Objectives Exchange 
, „ Box 24095 

4 Los Angelas, ^California -90024 

a 

♦ Jastak Associates, Inc. 

t 15J26 Gilpin Avenue 

Wilmington, Delaware 19806 

tern High School District 
2000 24th Street 

* Bakersfield, California 93301 
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McGrath publishing Company 
P.O. Box 9001 

Wilmington, North Carolina 28402 



McGraw-Hill Book Company 
8171 Redwood Highway 
Novato, California 94947 

New Mexico State Department of 

Education 
Monitor 

Education Building 
State Capitol 

Santa Fe, New Mexico 87501 

Noz^hern Illinois University 
Alan M. Voelker 
Curriculum & Instruction 
- De Xalb, Illinois 60115 

P«A«*H« Inc. 
Abbott Park Place 
Providence, Rhode Island 

Paul S. Amidon & Associates, Inc. 

1966 Benson Avenue 

St. Paul, Minnesota 55116 

Prep. Inc. 
j 1575 Parkway Avenue' 
Trenton, New Jersey 08628 

PRO-ED ' 

333 Perry. Brooks Building 
Austin, Texas 78701 

The Psychological Corporation 
Harcourt., Brace, Jovanovich, 

Publishers 
757 Third Avenue 
New York; New York 10017 

Richard L. Zweig, Associates, Inc. 
20800 Beach Blvd. 

Huntington Beach, California 92648 

Riverside Publishing Company 
*1919 South Highland Avehue 
Lombard, Illinois 60148 

c 

SOI . ^institute 
214 Main Street , , 
. El Segundo, California 90245. 



«chol«tic Testing Servics 
480 Meyer Road 
••nsenville, Illinois 60106 

******* toting s^vic." 
480 Meyer ^d 
•ineenville, Illinois 60106 

i55 N. Hacker Drive 
Chicago, Illinois 60606 

Scott, Foresman and Company' 
1900 «a«t Laka Avanua 
Glenview, lllinoia 60025 

l£i a0i ' Press 
Carbondale, ^Jllinoi. 62901 ' 

SITE ReSOUrce C «»ter 

s. ftrariKUn Street - 

Juneau, Alaska 99801 

Stanford University Press 
Stanford, California 94305 . 

Steck-Vaugh Company 

**0. Box 2028 \ 

Austin, Texas 78768 

Stoelting Company 
1350 8. Kostner 
Chicago, lllinoia 60523 



ttnited States Department of Labor 
Bureau of Labor Statistics 
1515 Broadway 
Mew fork. New York 10036 

0.S. Department of Labor 
.MANPOWER .Administration 
Washington^ D.C. 20210 

Western Psychological Services 
12031 Wilshire Blvd. 

Angeles, California 90025 
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